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MESSAGE FROM THE DIRECTOR 


Many people are asking how Alberta students 
are doing. There’s a mixed answer. Some are 
doing very well, according to the provincial 
achievement tests written last June. In two of 
the subjects tested, Grade 3 Science and 
Grade 6 Math, more students than expected 
performed at the standard of excellence. But 
the test results also show that the majority of 
students aren’t doing well enough. A smaller 
proportion than expected met the acceptable 
standard of performance. 


During the testing, we carried out a special 
study comparing students’ achievement now 
with that of students in 1987 and 1983. The 
results of the study are reported in Section 8 
of this report and show that students are 
doing better than in the previous years. In 
1991, for example, a higher percentage of 
Grade 3 Science students correctly answered 
process skills questions. The study was a 
complex one. It gave us useful information 
and helped us make some improvements in 
design for future studies. We would like to 
express our appreciation to the teachers, 
principals, and superintendents who helped 
us, and we look forward to the participation of 
schools in future projects. 


In this year’s report, our numbers express the 
proportion of students achieving standards on 
the two parts of each test—knowledge and 
process skills. To reach an acceptable stand- 
ard, students had to succeed on questions in 
both areas. This is harder than gaining a 
certain score based on questions from just 


any part of the test. We expect at least 80 per 
cent of all students within our defined popula- 
tion to achieve the acceptable standard, and 
we expect at least 10 per cent of all students 
within the same population to achieve the 
standard of excellence. 


This is the last year that we will report scores 
for the total test. 


For the first time, we have formally sought the 
views of the public on how students are doing. 
A public advisory meeting, attended by repre- 
sentatives of business, professional, and 
community groups, told us that our young 
people should be doing better than the test 
results showed. In the coming years, we will 
continue asking the public if student perform- 
ance is good enough. 


In the next round of achievement testing, we 
will collect information about what students 
know and can do in a few new performance 
areas: 


¢ problem-solving tasks in math and science 

¢ listening and viewing skills in language arts 
* participation skills in social studies 

We are looking forward to including in next 
year’s report information that will tell us more 
about students’ achievement. Such informa- 


tion helps us to focus on high expectations and 
on ensuring that students do their best. 


rank G. Horvath, Director 


Student Evaluation 
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SECTION 1 


SUMMARY OF ACHIEVEMENT TEST RESULTS 


Parents, educators, and the general public 
need to know how well Alberta students are 
achieving in relation to provincial curricular 
expectations. 


The 1991 achievement test results help dem- 
onstrate what is possible for grades 3, 6, and 
9 students to know and to do in Science, 
Mathematics, and Social Studies respectively. 


“Knowledge of what is possible produces new 
enthusiasm, raises sights, establishes new 
challenges and ultimately can improve per- 
sonal and societal performance.”—Learning 
Mathematics/Learning Science International 
Assessment of Educational Progress, Educa- 
tional Testing Service, February 1992. 


Guidelines for interpreting the 1991 results 
are given in Appendix C, and Appendix D 
provides answers to a number of questions 


frequently posed by parents. The process of 
developing the achievement tests is outlined 
in Appendix E. 


This section of the report describes certain 
broad characteristics of the student population 
who wrote the achievement tests and provides 
answers to the following questions: 


¢How many students wrote the achievement 
test for their grade and how many students 
did not write the achievement test for their 
grade? 


¢ What percentage of the students attained 
the acceptable standard according to provin- 
cial criteria? 


¢ What percentage of the students attained 
the standard of excellence according to 
provincial criteria? 


For the first time, results are reported in 

terms of three related but different stand- 
ards: curriculum standards, achievement 
standards, and assessment standards. 


Curriculum standards are the expected 
student learnings, sequenced into levels that 
are stated for each curriculum. They include 
specific statements of knowledge, skills, and 
attitudes against which student performance 
is to be judged. 


Achievement standards, usually expressed 
as percentages, state how many Alberta stu- 
dents are expected to reach a particular curri- 
culum level at a specific age or year in school. 
It is important to point out that this judg- 
ment is not a prediction of what percentage of 
students will actually achieve acceptable or 
excellent levels of performance but rather 
specifies the percentage of students at a given 
age or year in school who are expected to 


achieve the acceptable or excellent levels. 
These standards apply to school, jurisdiction, 
and provincial achievement. 


Assessment standards are the scores to be 
achieved by a student on a specific test or part 
of a test before the performance of that stu- 
dent is judged to be “acceptable” or “excellent” 
relative to the curriculum standards. Alberta 
Education reports the results for the 
achievement tests in relation to an acceptable 
standard and a standard of excellence level 
respectively. 


In previous years, assessment standards for 
the total test and for individual components 
were determined and reported. In 1991, in an 
effort to provide more meaningful and useful 
results to schools and jurisdictions, achieve- 
ment and assessment standards are reported 
for each major component of the tests and 
for both components together. 


Assessment and achievement standards for 
the 1991 achievement tests were determined, 
and subsequently recommended to the Direc- 
tor of Student Evaluation, by teams of cur- 
riculum specialists, test development special- 
ists, and experienced subject classroom 


teachers who applied standard-setting proce- 
dures to the tasks under the guidance of the 
Analytic Services Unit of the Student Evalua- 
tion Branch. An outline of the processes 
followed is provided in Appendix A. 


STUDENT POPULATIONS 


In June 1991, principals reported a total 
population of 108 334 students in the grades 
tested. There were 39 216 students in 
Grade 3, 36 565 students in Grade 6, and 

32 553 students in Grade 9. Of that total 
population, 97 858 wrote achievement tests. 
The remaining 10 476 students were ex- 
empted from writing or were absent the day 


Figure 1-1 


the tests were administered, as shown in 
Figure 1-1. The tests were administered in 
several special forms in addition to the regu- 
lar form. The information collected from the 
special forms was used to investigate student 
achievement over time and the effects of the 
language of testing. 


STUDENTS WRITING ACHIEVEMENT TESTS 
June 1991 


Science (Gr. 3) 


Mathematics (Gr. 6) 


Students writing the test* 


Social Studies (Gr. 9) 


Students absent or exempt 


*The number of students writing the tests includes students in the regular English program as well as 
Francophone and French Immersion students who wrote the achievement test in either English or French. 


Provincial results are based on test scores achieved by students who were in the regular English program, which 
is defined in the first footnote for tables 2-2, 3-3, and 4-3. Results for Francophone and French Immersion stu- 
dents are reported directly to participating schools and jurisdictions and are not calculated in provincial results. 


OBSERVATIONS ON STUDENT PARTICIPATION 


The student participation rate was highest in 
Grade 6 (92.5%) and lowest in Grade 3 
(88.8%) when calculated from the numbers 
reported on the Principal’s Reports submitted 
after each achievement test administration. 
However, the lower participation rate in 
Grade 3 Science may be because there was no 
French translation of the test for those stu- 
dents instructed in French. In terms of 
absolute numbers, more Grade 3 students 
(34 825) wrote their test in Science than did 
Grade 6 students in Mathematics (33 814) 
and Grade 9 students in Social Studies 

(29 219). 


When participation rates are calculated from 
the September 30, 1990, enrolment figures, 
however, another picture emerges. In Grade 3 
Science, the participation rate decreases to 
84.9%; in Grade 6 it falls to 91.1%; and in 
Grade 9, it is reduced to 82.3%. In terms of 
numbers, approximately 1 785 Grade 3 stu- 
dents, 549 Grade 6 students, and 2 959 Grade 
9 students were not accounted for administra- 
tively at testing time. Obviously, administra- 
tive accounting must be enhanced before the 
next round of testing. 


Details of student participation at each grade 
level, including the number of students who 
were absent or exempted, are presented in 
sections 2, 3, and 4 of this report. 


RESULTS IN RELATION TO STANDARDS FOR STUDENTS WHO WROTE 
THE ACHIEVEMENT TESTS 


For the 1991 results, the focus of reporting of 
student achievement is on the major compo- 
nents of the test. Results are reported in 
relation to the standards established for 
individual major components as well as for 
both components. 


Acceptable Standard 


Each major component 


Both major components 


Based on discussions with educators, test 
development specialists, and curriculum 
specialists, and on our experience with meas- 
uring student achievement according to the 
expectations in the Program of Studies, 
achievement standards were confirmed as 
follows: 


Standard of Excellence 


Achievement standards were also confirmed 
for the total test: 


Acceptable Standard 85% 
Standard of Excellence 15% 


Standards reflect expectations for the regular 
English program students and not for the 


total population—which, for example, would 
include special needs students. 


Figures 1-2 and 1-3 present the percentage of 
students who achieved the acceptable stand- 

ards and the standards of excellence based on 
their scores on both components of the test. 


Figure 1-2 


PERCENTAGE OF STUDENTS ACHIEVING ACCEPTABLE 
STANDARDS OR HIGHER* ON BOTH COMPONENTS OF THE TEST 
June 1991 


100 


Science (Gr. 3) Mathematics (Gr. 6) Social Studies (Gr. 9) 
WN = 33 837) (N = 31 930) (N = 27 646) 


*Includes students achieving the standard of excellence 
**80% of students were expected to achieve the acceptable standard or higher on both components of the test. 


Figure 1-3 


PERCENTAGE OF STUDENTS ACHIEVING STANDARDS 
OF EXCELLENCE ON BOTH COMPONENTS OF THE TEST 
June 1991 


Science (Gr. 3) Mathematics (Gr. 6) Social Studies (Gr. 9) 
(N = 33 837) (N = 31 930) (N = 27 646) 


*10% of students were expected to achieve the standard of excellence on both components of the test. 


Figures 1-4 and 1-5 present the percentage of ard or the standard of excellence based on 
students who achieved the acceptable stand- their total test score. 


Figure 1-4 


PERCENTAGE OF STUDENTS ACHIEVING ACCEPTABLE 
STANDARD OR HIGHER* ON TOTAL TEST SCORE 
June 1991 


Science (Gr. 3) Mathematics (Gr. 6) Social Studies (Gr. 9) 
(N = 33 837) (N = 81 930) (N = 27 646) 


*Includes students achieving the standard of excellence 
**85% of students were expected to achieve the acceptable standard or higher on the total test. 


Figure 1-5 


PERCENTAGE OF STUDENTS ACHIEVING STANDARD 
OF EXCELLENCE ON TOTAL TEST SCORE 
June 1991 


Science (Gr. 3) Mathematics (Gr. 6) Social Studies (Gr. 9) 
(N = 33 837) (N = 831 980) (N = 27 646) 


*15% of students were expected to achieve the standard of excellence on the total test. 


OBSERVATIONS ON GENERAL RESULTS 


In Grade 3 Science, results based on both 
components of the test revealed that the 
percentage of students achieving standards 
was lower than expected for the acceptable 


standard but slightly higher than expected for 


the standard of excellence. This pattern was 
similar to that of results based on the total 
test score, where levels of performance were 
lower than expected for the acceptable stand- 
ard but slightly higher than expected for the 
standard of excellence. 


Results for Grade 6 Mathematics exhibited a 
similar pattern. However, a smaller percent- 


age of Grade 6 students than Grade 3 students 


achieved the acceptable standard but a larger 
percentage of Grade 6 students than Grade 3 


students achieved the standard of excellence. 
This is true for results based on both compo- 
nents of the test as well as for results based 
on the total test score. 


For Grade 9 Social Studies, results were 
disappointing for both levels of performance. 
The percentage of students achieving stand- 
ards was lower than expected for the accept- 
able standard as well as for the standard of 
excellence. 


In conclusion, results showed that students at 
all three grades need improvement to achieve 
the acceptable level and Grade 9 students also 
need improvement to achieve the standard of 
excellence. 


SECTION 2 


GRADE 3 SCIENCE 

GENERAL DESCRIPTION 
The Grade 3 Science Achievement Test con- ¢ How many Grade 3 students wrote each 
sisted of 50 multiple-choice questions. The form of the test or were absent and exempt? 
time allotted for writing the test was 25 ¢ What percentage of Grade 3 students 
minutes for each of two sections. Statistics for attained the acceptable standard in 
the total test and for the components are Science according to provincial criteria? 
based on the results achieved by 33 837 ¢ What percentage of Grade 3 students 
students (30 427 wrote the regular form, and attained the standard of excellence in 
3 410 participated in the achievement-over- Science according to provincial criteria? 
time study, as shown in Table 2-2). This ¢ What did Grade 3 students know and what 
section of the report provides answers to the could they do in Science? 
following questions: ¢ What parts of the Science curriculum 


caused Grade 3 students difficulty? 


SUMMARY OF RESULTS 


RESULTS IN RELATION TO STANDARDS 


Results show that 70.1% of students who wrote the test achieved the acceptable standards on 
both components and 11.0% achieved the standards of excellence on both components. These 
results were lower than expected for the acceptable standards but slightly higher than expected 
for the standards of excellence. 


AVERAGE SCORE 


The average total test score was 68.4%, with a standard deviation of 18.2. The average total 
test raw score was 34.2 marks out of a possible 50, with a standard deviation of 9.1. 


CONTENT OF THE TEST 


The Grade 3 Science Achievement Test was ¢ Subject Matter 

designed to reflect the Grade 3 Science Cur- ¢ Process Skills 

riculum Specifications (revised May 1986) 

that were developed from the revised 1984 The subject matter component consisted of 


Science Program of Studies. The scope of the questions associated with the major themes of 
test was limited to curriculum objectives that science in Division I. The process skills 

could be efficiently measured on a paper and component consisted of questions integrated 
pencil test. As a result, questions on the test with subject matter and questions independ- 
were drawn from the content of the two major _ ent of subject matter. 

components in the core program: 


Questions on the Grade 3 Science Achievement Test measured student achievement in two 
cognitive levels: 


¢ Knowledge—recognize or recall ideas, ¢ Comprehension and Application—demon- 
terminology, facts, conventions, methods of strate an understanding of the concepts and 
inquiry, principles, generalizations, theo- skills, and apply appropriate methods and 
ries, and concepts ideas to a new situation 


ACHIEVEMENT TEST BLUEPRINT 


Table 2-1 presents the blueprint used to develop the Grade 3 Science Achievement Test. Clas- 
sification of each question by component, cognitive level, and subtest is indicated in the table. 


Table 2-1 
Grade 3 Science 
Achievement Test Blueprint 
June 1991 


| Component | | Cognitive Level Level 


Process <_* 
Skills Matter Total Knowledge | Application | Total 


Matter and Energy | 4,5, 6, 1, 26, 27 2, 3, 4, 5, 
15 [30%] 6, 7, 8, 28, 
9, 10, 33 
15 [30%] 


29, 30, 31, 
Earth, Space, and 16, 42 17,41 4 [8%] 16, 41, 42 
Time 


32 
Process Skills , 19, 18, 45,46 | 19, 20, 21, 
Independent of , 21, 16 [82%] 22, 23,24, | 16 [32%] 
Prescribed , 23, 25, 43, 44, 
Subject Matter 2s, 47, 48, 49, 
, 44, 50 


36 [72%] | 14 [28%] | 50 [100%] | 10 (20%) | 40[80%) | 50 [100%] 


Curriculum 
Component 


15 [80%] 


11, 12, 18, 
14, 15, 34, 
35, 36, 37, 
38, 39, 40 


Living Things and 
Environment 


15 [30%] 


STUDENT PARTICIPATION 


In June 1991, principals reported a total absent, and who were exempted. Results for 
population of 39 216 students in Grade 3. students in French Immersion or Francoph- 
Table 2-2 presents the number and percent- one programs are reported separately to 

age distribution of students who wrote the participating schools and jursidictions. 


Grade 3 Science Achievement Test, who were 


Table 2-2 
Grade 3 Science 
Student Participation 
June 1991 


Number Percentage 
Category of Students of Students 


Other Students Who Wrote:*** 


Francophone Students—Wrote in French 
Francophone Students—Wrote in English 
French Immersion Students—Wrote in French 
French Immersion Students—Wrote in English 
Language-of-Testing Students—Wrote in French 
Language-of-Testing Students—Wrote in English 


Students Absent 


Students Exempted from Writing 
Categories of Exemption: 


. Special Needs Students 

. Subject Was Not Taught This Term 

. ESL Students 

. Language of Instruction Was Not English 

. Other (as approved by the Superintendent) 


Total Principals’ Reported Population: 
Test Day, June 1991 


Grade 3 Enrolment: September 30, 1990 41 001 


*Provincial results are based on test scores achieved by students who were in the “regular” (Mathematics or Science 
or Social Studies) programs. Such students are defined as those who were instructed in English and wrote the 
English form of the achievement test, and those students who were instructed in a language other than English or 
French and wrote the English form of the test. These students are indicated by the shaded area of the table. 


**Regular Form refers to the unmodified 1991 test. The modified tests are described in Section 8 of this report. 


***Results for Francophone or French Immersion students or for students participating in the language-of-testing 
study are reported directly to participating schools and jurisdictions and are not calculated in the provincial 
results. 9 


RESULTS IN RELATION TO STANDARDS 


Figures 2-1 and 2-2 and Table 2-3 show the percentage of students achieving the acceptable 
standards and the standards of excellence on components of the test. The levels of performance 
were higher than expected for the standards of excellence and lower than expected for the 
acceptable standards. 
Figure 2-1 
GRADE 3 SCIENCE 
PERCENTAGE OF STUDENTS ACHIEVING ACCEPTABLE 
STANDARDS OR HIGHER ON COMPONENTS OF THE TEST 
June 1991 


Subject 
Skills Matter Components 


WY Achievement Standards* 


(See Appendix A) 


*the percentage of students in the province expected to meet the acceptable standards or higher 
**the percentage of students in the province who met the acceptable standards or higher 


Actual Results** 


Figure 2-2 


GRADE 3 SCIENCE 
PERCENTAGE OF STUDENTS ACHIEVING STANDARDS OF 
EXCELLENCE ON COMPONENTS OF THE TEST 
June 1991 


25 


20 


15 
% 


10 


Subject Both 
Skills Matter Components 
Y Achievement Standards* |Actual Results** 
(See Appendix A) 


*the percentage of students in the province expected to meet the standards of excellence 
**the percentage of students in the province who met the standards of excellence 
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Table 2-3 
Grade 3 Science 
Students Achieving Standards on Components of the Test 
June 1991 


Provincial | Provincial 


Possible 
Reporting Category Score 


Standard of Excellence 
Both Major Components*** 36 + 14 
Process Skills Only 
Subject Matter Only 


Acceptable Standard 
or Higher 
Both Major Components*** 
Process Skills Only 
Subject Matter Only 


Below Acceptable Standard 
on Both Components 


*The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the 
lowest score students must achieve for their performance to be judged “acceptable” and/or “excellent” in relation 
to curricular expectations. See Appendix A. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the 
Provincial Assessment Standard. See Appendix A. 


***In previous years, a standard for the total test was reported. An individual could meet the standard for the test 
overall but might not achieve the standard for both components of the test. The new method of reporting allows 
for a more meaningful interpretation of the results. To allow comparison with results from previous years, the 
total test score is reported in Table 2-4. 


541s 


Table 2-4 shows the percentage of students achieving the acceptable standard and the stand- 
ard of excellence on the total test. These levels of performance exceeded the expectations for 
the standard of excellence but were lower than expected for the acceptable standard. 


Table 2-4 
Grade 3 Science 
Students Achieving Standards on the Total Test 
June 1991 


Possible 
Score 


Reporting Category 


Total Test: 
Standard of Excellence 


Acceptable Standard 
or Higher 


Provincial 
Maximum | Assessment |Achievement 
Standard* 
(Raw Score) 


50 44 15 5076 | 5606 16.6 
50 27 85 28 761 | 26 710 78.9 


Provincial 


Standard** 
(Per Cent) 


*The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the 
lowest score students must achieve for their performance to be judged “acceptable” and/or “excellent” in relation to 


curricular expectations. See Appendix A. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the Provin- 


cial Assessment Standard. See Appendix A. 


It should be noted that the actual percentages 
for Grade 3 Science (16.6 and 78.9) are based 
on the 33 837 students in the regular pro- 
grams who wrote the test. 


If, however, the percentages are based on the 
total population reported by principals 
(39 216), less the French Immersion and 
Francophone students who wrote the test 
(988), the percentages would be: 
14.7% achieving the standard of excellence 
69.9% achieving the acceptable standard 


If the percentages are based on the Septem- 
ber 30, 1990, Grade 3 enrolment (41 001), less 
the French Immersion and Francophone 
students who wrote the test (988), the per- 
centages would be: 
14.0% achieving the standard of excellence 
66.8% achieving the acceptable standard 


It is emphasized that the above percentages, 
based on the total population and enrolment 
figures, present the worst possible scenario. 


It is highly unlikely that none of the students 
who was absent, exempt, or not accounted for 
would have achieved standards. The absence 
of information on these students is nonethe- 
less problematic. 


The number of students achieving the accept- 
able standard and the standard of excellence 
for each jurisdiction was analysed to deter- 
mine whether jurisdictions were below, meet- 
ing, or exceeding provincial achievement 
standards. Jurisdictions classified as meeting 
provincial achievement standards were those 
for which the difference between the actual 
number of students and the expected number 
of students at or above standards was not 
statistically significant. A 95% confidence 
interval was used; this criterion means that 
differences are only reported when there is a 
5% or smaller probability that a difference of 
that size could occur by chance. The results 
are reported in Tables 2-5 and 2-6. The 
percentages in the tables are based on 200 
jurisdictions (including private schools). 
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Table 2-5 
Grade 3 Science 
Percentage Distribution of Jurisdictions* Meeting 
Provincial Achievement Standards on Components of the Test 
June 1991 


Jurisdictions Not Jurisdictions Jurisdictions Ex- 

Meeting Provincial | Meeting Provincial | ceeding Provincial 

Achievement Stan-} Achievement Stan-| Achievement Stan- 
Component dard (Per Cent) dard (Per Cent) dard (Per Cent) 


Standard of Excellence 
Both Major Components 


Process Skills Only 
Subject Matter Only 


Acceptable Standard or Higher 
Both Major Components 
Process Skills Only 
Subject Matter Only 


Table 2-6 
Grade 3 Science 
Percentage Distribution of Jurisdictions* Meeting 
Provincial Achievement Standards on the Total Test 
June 1991 


Jurisdictions Jurisdictions 
Not Meeting | Jurisdictions | Exceeding 
Provincial Meeting Prov- | Provincial 
Achievement | incial Achieve- | Achievement 
Standard ment Standard | Standard 
Reporting Category (Per Cent) (Per Cent) (Per Cent) 


Total Test: 


Standard of Excellence 


Acceptable Standard or Higher 


* Jurisdictions with fewer than six students are excluded because the statistical significance of the difference 


between the number actually meeting the standard and the number expected to meet the standard when 
calculated and reported is not educationally meaningful. 


ot ae 


REPORTING CATEGORIES 


Table 2-7 shows the total marks possible and _ jurisdiction and school reports. Variations in 
the provincial raw score results for the patterns of students’ responses can help indi- 
reporting categories of the Grade 3 Science cate strengths and weaknesses in local educa- 


Achievement Test. 


tional programs. 


It is important to stress that the averages in Statistics presented in this table are based on 


the various reporting categories cannot be results achieved by 31 596 students (those who 
directly compared with one another. Rather, wrote the regular form and those who wrote 
the results shown in Table 2-7 can best be Form G). 


used in conjunction with parallel tables in the 


Table 2-7 
Grade 3 Science 


Raw Score Results by Reporting Category 


Reporting Category 
Total Test 


Major Component: 
Process Skills 
Subject Matter 


Curricular Elements: 
Matter and Energy 
Living Things and Environment 
Earth, Space, and Time 
Process Skills as Content 


Cognitive Level: 
Knowledge 
Understanding and Application 


June 1991 


Maximum 
Possible Raw Score Raw Score 
Raw Score Average Standard Deviation 
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PERCENTAGE OF STUDENTS CHOOSING EACH ALTERNATIVE 


Table 2-8 shows the percentage of students 
who chose each alternative (A, B, C, and D) 
for each multiple-choice question. The correct 
response (key) for each question is also 
identified. The results shown in Table 2-8 can 
best be used in conjunction with the parallel 
tables in the jurisdiction and school reports. 


Variations in patterns of students’ responses 
to questions can help to indicate strengths and 
weaknesses in local educational programs. 


Statistics presented in this table are based on 
results achieved by 31 596 students (those who 
wrote the regular form and those who wrote 
Form G). 


Table 2-8 
Grade 3 Science 
Provincial Results for Individual Multiple-Choice Questions* 
June 1991 


Distribution of 
Responses (%) 


Distribution of 
Responses (%) 


is not included. This category is less than 8% for any of these questions. 
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GRADE 3 SCIENCE EXAMINER'S OBSERVATIONS 


The following is a discussion of specific areas of strength and of difficulty for Grade 3 students. 
Sample questions are provided to highlight the strengths and weaknesses of students achiev- 
ing at the acceptable standard and at the standard of excellence. For each sample question, 
the circle beside or under the correct answer has been filled in. The percentage of students 
choosing each alternative is given beside each circle. 


Acceptable Standard—Sample Questions and Commentary 


7. If the bottie is heated, eam 
what will happen to the 5 balloon 
balloon? bottle 


40. Which group of plants and animals are found in an 
Alberta forest community? 


19. Look at this picture. 28. To conserve energy, Jean could 


6.8 O turn up the heat in his house 
3.1 O leave the refrigerator door open 
86.6 @ turn off ali the lights when he 
An OBSERVATION that can be made about this goes to bed 
pencil is thet it has 
2.2 O leave the television on when he 
54.1 ® apointed end goes out to play 
19.7 O just been sharpened 
11.9 O been used for writing 


10.4 O apink eraser on the end 


What did students know and what could they do? 


Generally, these students had an understanding of basic 
process skills. They could classify common animals 
based on similar physical characteristics; they could 
identify similarities presented in a graph or achart. By 
observing, they could predict the next step in a sequence 
of events and could make simple interpretations. 


Specifically, these students knew that when materials 
are heated, they occupy more space and can increase in 
size (question 7). As well, they knew that common 
objects made of specific materials may or may not have 
the capacity to float on water or be attracted by a mag- 
net. They also could identify some common plant and 
animal life of Alberta (question 40) and knew the impor- 
tance of conserving energy in a home (question 28). 
Students knew that moss and mushrooms are types of 
plants that grow on decaying logs, and that birds, fish, 
and reptiles hatch from eggs. They understood the 
energy (food) chains that exist between plants and ani- 
mals as well as the life cycle of a plant. 


“16. 


8. This Is the temperature today. | 


What would be BEST to wear outside today? 


Rigen 
Raincoat and rubber boots 


oO 39 


£49398 8 


Jogging suit and running 
shoes 


O 14.2 


What did students find difficult? 


These students had difficulty applying their science 
knowledge to new situations or to contexts that went 
beyond familiar plants and animals. Generally, they 
could not sort out extraneous information or apply 
their observation skills objectively. 


In question 8, for example, students achieving at the 
acceptable standard had difficulty making the neces- 
sary connections between temperature measure- 
ments and the conditions that exist in the environ- 
ment. In question 19 (see the previous page), stu- 
dents were asked to make a direct observation about 
the pencil in the picture. Their responses indicated 
that they tended to rely on what they knew a pencil 
looked like rather than on what they could see in the 
picture. 


Standard of Excellence—Sample Questions and Commentary 


13, Tom predicted that radish seeds grow better in sand 


than in garden soil. 


In each pot, he planted six radish seeds. 


Garden soil 


What did students know and what could they do? 


Students achieving at the standard of excellence 
knew that a specific unit of measurement is neces- 
sary to make accurate measurements and that the 
senses of touch and sight can be used to estimate the 
length of an object. They also knew which variables 
to control when experimenting with plant growth 
(question 13). These students also knew that metals 
expand when heated and that the phase changes of 
water can be reversed whereas the process of burn- 
ing paper cannot. In addition, they understood the 
interrelationship that exists between living things 
and their environment (question 25) and how sym- 
bols can be used to represent changes of events that 
occur in nature. 


17s 


25. Some trees in an area were cut down to make room for a 
baseball diamond. 


—— eae a Me Ri 
Aap Mises 


=hien.. : we es 
= 


What is the MOST likety result of this change? 
8.5 O There are more shady places. 
68.2 @ There are fewer birds in the area. 
9.7 © Children have less room to play. 
6.4 © More squirrels have built homes In the area. 
24, Five students recorded the number of floor hockey 
games each of them played in one year. 
or EOOR HOCKEY GAMES PLAYED 


2 
Dana Ted Tim Stan Jackie 
Players 


Which etudent played 40 more games than Jackie? 


63.1 @ Ted 
5.6 O Dana 
20.5 © Tim 


5.3 O Stan 


These students had good observation and inference 
skills. They could read an outdoor thermometer, and 
they could order objects from lightest to heaviest based 
on their mass. These students could also classify 
unfamiliar living and nonliving things based on their 
observations of physical characteristics, apply general 
knowledge to solve novel problems, transfer new 
learnings gained from interpreting graphs (question 
24), and analyse new situations. 


What did students find difficult? 


Analysis indicates that students achieving at the 
standard of excellence did not have any apparent 


difficulties with Grade 3 Science knowledge or process 
skills. - 


Teachers' Review of the Test 


The Grade 3 Science teachers who reviewed the test 
felt that it was fair but tough. As well, they felt that 
the concepts and skills tested reflected the learning 
expectations of the Division I Science Program very 
well. Teachers also liked the use of questions that 
required students to think and apply their knowledge 
of science to the world they experience beyond the 
classroom. 
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SECTION 3 


GRADE 6 MATHEMATICS 


GENERAL DESCRIPTION 


The Grade 6 Mathematics Achievement Test 
was divided into three booklets. Booklet 1 
contained 36 questions, and students were 
allowed to use a calculator; Booklet 2 
contained 19 questions and students were not 
allowed to use a calculator; Booklet 3 
contained five 2-minute tests of 48 number 
fact questions each, and students were not 
allowed to use a calculator. The time allotted 
was 55 minutes for Booklet 1 and 30 mintues 
for Booklet 2. 


Statistics for the total test and for components 
in Booklets 1 and 2 are based on the results 
achieved by 31 930 students (28 649 wrote the 
regular form, and 3 281 participated in the 
achievement-over-time study, as shown in 
Table 3-3). 


This section of the report provides answers to 
the following questions: 


¢ How many Grade 6 students wrote each 
form of the test or were absent and exempt? 

¢ What percentage of Grade 6 students 
attained the acceptable standard in 
Mathematics according to provincial 
criteria? 

¢ What percentage of Grade 6 students 
attained the standard of excellence in 
Mathematics according to provincial 
criteria? 

¢ What did Grade 6 students know and what 
could they do in Mathematics? 

¢ What parts of the Mathematics curriculum 
caused Grade 6 students difficulty? 


SUMMARY OF RESULTS FOR BOOKLETS 1 AND 2 


RESULTs IN RELATION TO STANDARDS 


Results show that 68.9% of students who wrote the test achieved the acceptable standards on 
both components and 14.3% achieved the standards of excellence on both components. These 
results were lower than expected for the acceptable standards but higher than expected for the 


standards of excellence. 


AVERAGE SCORE 


The average total test score for booklets 1 and 2 was 68.2%, with a standard deviation of 19.1. 
The average raw score was 37.5 marks out of a possible 55, with a standard deviation of 10.5. 


CONTENT OF THE TEST 


The Grade 6 Mathematics Achievement Test 
was designed to reflect the Grade 6 
Mathematics Curriculum Specifications 
(revised May 1986) that were developed from 
the revised 1982 Mathematics Program of 
Studies. The scope of the test was limited to 
curriculum objectives that could be efficiently 
measured on a paper and pencil test. 
Questions on the test were drawn from the 
content of two major components in the core 
program: 
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¢ Subject Matter 
¢ Problem Solving 


The subject matter component consisted of 
questions associated with the six content 
areas: numeration, operations and proper- 
ties, measurement, geometry, graphing, and 
problem-solving strategies. The problem- 
solving component consisted of questions 
integrated with content areas and questions 
independent of subject matter. 


Questions on the Grade 6 Mathematics from concrete to pictorial to symbolic repre- 
Achievement Test measured student achieve- sentations or translate in the reverse order 
ment at three cognitive levels: 

¢ Application and Problem Solving—solve 


° Knowledge—recognize or recall facts, defini- problems utilizing learned skills and con- 
tions, rules, procedures, and perform rou- cepts; recognize patterns and relationships; 
tine manipulation compare, interpret, and analyse data by 

* Comprehension—demonstrate understand- applying learned skills and knowledge. 


ing of principles and concepts; translate 


ACHIEVEMENT TEST BLUEPRINT 


Table 3-1 presents the blueprint used to develop booklets 1 and 2 of the Grade 6 Mathematics 
Achievement Test. Classification of each question by component, cognitive level, and subtest is 


indicated in the table. 
Table 3-1 


Grade 6 Mathematics 
Achievement Test Blueprint, Booklets 1 and 2 
June 1991 


Component Cognitive Level 
Application 
Curricular Subject and Problem 
Elements i Matter Total 1 


Numeration 4, 53, 54 2 
6 


, 53, 54, | 1, 2, 3, 13 [23.6%]}1, 2, 14, 13 [23.6%] 
55 5, 6, 14, 38, 39 
15, 38, 39 


Operations 20, 27, 36,| 29, 37, 40, | 16 [29.1%]}42 16 [29.1%] 
and Properties | 41, 47, 48,| 42, 43, 44, 
50 45, 46, 49 


Measurement | 16,28, 35,!7,8,11, |11(20.0%1|7,12,13, |8,11 —_|16, 28,35, | 11 [20.0%] 
51.52 | 12, 13, 17 17 51, 52 
ae 7 9, 30,31, | 6[10.9%] 31,34 132,33 6 [10.9%] 

33, 34 


Graphing 21, 22, 23, 6 [10.9%] 24, 26 21, 22, 23, 6 [10.9%] 
24, 25 25 
Problem-Solving} 10, 18, 19 3 [5.5%] 10, 18, 19 3 [5.5%] 
Strategies 


Total = 25 25 (45.5%) 5%] |30 30 [55.5%] | 5%) | 55 55 (100% | 12 12 [21.8%] |19 [34.5%] 8%) {19 [34.5%] | 24 143.6%] | | 24 143.6%] | 6%] | 55 } 55 (100%) | 
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Table 3-2 presents the blueprint used to develop Booklet 3 of the Grade 6 Mathematics 
Achievement Test. This booklet consisted of five timed tests, measuring student performance 
in operations involving sums and minuends to 18, and products and dividends to 81. 


Table 3-2 
Grade 6 Mathematics 
Achievement Test Blueprint, Booklet 3 
Number Facts: Speed and Accuracy Tests 
June 1991 


Number 
of 
Operations Questions 


Addition 


Subtraction 


Multiplication 


Division 


Mixed Operations 


-21- 


STUDENT PARTICIPATION 


In June 1991, principals reported a total who were absent, and who were exempted. 
population of 36 565 students in Grade 6. Results for students in French Immersion or 
Table 3-3 presents the number and percent- Francophone programs are reported sepa- 
age distribution of students who wrote the rately to participating schools and jurisdic- 
Grade 6 Mathematics Achievement Test, tions. 
Table 3-3 
Grade 6 Mathematics 
Student Participation 
June 1991 


Number Percentage 
Category of Students of Students © 


Other Students Who Wrote:*** 


Francophone Students—Wrote in French 
Francophone Students—Wrote in English 
French Immersion Students—Wrote in French 
French Immersion Students—Wrote in English 
Language-of-Testing Students—Wrote in French 
Language-of-Testing Students—Wrote in English 


Students Absent 


Students Exempted from Writing 
Categories of Exemption: 


. Special Needs Students 

. Subject Was Not Taught This Term 

. ESL Students 

. Language of Instruction Was Not English 

. Other (as approved by the Superintendent) 


Total Principals’ Reported Population: 
Test Day, June 1991 


Grade 6 Enrolment: September 30, 1990 37 114 


* Provincial results are based on test scores achieved by students who were in the “regular” (Mathematics or Science or 
Social Studies) programs. Such students are defined as those who were instructed in English and wrote the English 
form of the achievement test, and those students who were instructed in a language other than English or French and 
wrote the English form of the test. These students are indicated by the shaded area of the table. 


» ** Regular Form refers to the unmodified 1991 test. The modified tests are described in Section 8 of this report. 


***Results for Francophone or French Immersion students or for students participating in the language-of-testing study 
are reported directly to participating schools and jurisdictions and are not calculated in the provincial results. 
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RESULTS FOR BOOKLETS 1 AND 2 


Figures 3-1 and 3-2 and Table 3-4 show the percentage of students achieving the acceptable 
standards and the standards of excellence on components of the test. The levels of performance 


were higher than expected for the standards of excellence but lower than expected for the 
acceptable standards. 


Figure 3-1 


GRADE 6 MATHEMATICS 
PERCENTAGE OF STUDENTS ACHIEVING ACCEPTABLE 
STANDARDS OR HIGHER ON COMPONENTS OF THE TEST 
June 1991 


Subject 
Matter Components 


Y Achievement Standards* 
(See Appendix A) 


*the percentage of students in the province expected to meet the acceptable standards or higher 
**the percentage of students in the province who met the acceptable standards or higher 


Figure 3-2 


=| Actual Results** 


GRADE 6 MATHEMATICS 
PERCENTAGE OF STUDENTS ACHIEVING STANDARDS 
OF EXCELLENCE ON COMPONENTS OF THE TEST 
June 1991 


LLL 


Li 


Problem a Both 
Solving Matter Components 


G Achievement Standards* 
(See Appendix A) 
*the percentage of students in the province expected to meet the standards of excellence 
**the percentage of students in the province who met the standards of excellence 
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Actual Results** 


Table 3-4 
Grade 6 Mathematics: Booklets 1 and 2 
Students Achieving Standards on Components of the Test 
June 1991 


Provincial | Provincial 


Reporting Category 


Standard of Excellence 
Both Major Components*** 
Problem Solving Only 
Subject Matter Only 


Acceptable Standard 
or Higher 
Both Major Components*** 25 + 30 13 +17 
Problem Solving Only 25 13 
Subject Matter Only 30 


Below Acceptable Standard N/A 
on Both Components 


* The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the 
lowest score students must achieve for their performance to be judged “acceptable” and/or “excellent” in relation to 
curricular expectations. See Appendix A. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the Provincial 
Assessment Standard. See Appendix A. 


***In previous years, a standard for the total test was reported. An individual could meet the standard for the test 
overall but might not achieve the standard for both components of the test. The new method of reporting allows for a 
more meaningful interpretation of the results. To allow comparison with results from previous years, the total test 
score is reported in Table 3-5. 
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i 


Table 3-5 shows the percentage of students achieving the acceptable standard and the stand- 
ard of excellence on the total test. These levels of performance were higher than expected for 
the standard of excellence but were lower than expected for the acceptable standard. 


Table 3-5 
Grade 6 Mathematics: Booklets 1 and 2 
Students Achieving Standards on the Total Test 
June 1991 


Possible 
Score 


Reporting Category 


Total Test: 
Standard of Excellence 


Acceptable Standard 
or Higher 


Provincial 
Maximum |Assessment | Achievement 
Standard* 
(Raw Score) 


Provincial 


Standard** Actual | Actual 


(Per Cent) 


Expected 
Number | Number |Per Cent 


55 48 15 4 789 6 566 20.6 
55 30 85 27140 | 24118 75.5 


*The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the 
lowest score students must achieve for their performance to be judged “acceptable” and/or “excellent” in relation 
to curricular expectations. See Appendix A. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the Provin- 


cial Assessment Standard. See Appendix A. 


It should be noted that the actual percentages 
for Grade 3 Mathematics (20.6 and 75.5) are 
based on the 31 930 students in the regular 
programs who wrote the test. 


If, however, the percentages are based on the 
total population reported by principals 
(36 565), less the French Immersion and 
Francophone students who wrote the test 
(1 884), the percentages would be: 
18.9% achieving the standard of excellence 
69.5% achieving the acceptable standard 


If the percentages are based on the Septem- 
ber 30, 1990, Grade 6 enrolment (37 114), less 
the French Immersion and Francophone 
students who wrote the test (1 884), the 
percentages would be: 
18.6% achieving the standard of excellence 
68.5% achieving the acceptable standard 


It is emphasized that the above percentages, 
based on the total population and enrolment, 


present the worst possible scenario. It is highly 
unlikely that none of the students who was absent, 
exempt, or not accounted for would have achieved 
standards. The absence of information on these 
students is nonetheless problematic. 


The number of students achieving the acceptable 
standard and the standard of excellence for each 
jurisdiction were analysed to determine whether 
jurisdictions were below, meeting, or exceeding 
provincial achievement standards. Jurisdictions 
classified as meeting provincial achievement 
standards were those for which the difference 
between the actual number of students and the 
expected number of students at or above standards 
was not statistically significant. A 95% confidence 
interval was used; this criterion means that differ- 
ences are only reported when there is a 5% or 
smaller probability that a difference of that size 
could occur by chance. The results are reported in 
tables 3-6 and 3-7. The percentages in the 
tables are based on 202 jurisdictions (includ- 
ing private schools). 
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Table 3-6 
Grade 6 Mathematics: Booklets 1 and 2 
Percentage Distribution of Jurisdictions* Meeting 


Provincial Achievement Standards on Components of the Test 


June 1991 


Jurisdictions 
Not Meeting 
Provincial 
Achievement 
Standard 
(Per Cent) 


Jurisdictions 
Meeting Prov- 
incial Achieve- 
ment Standard 
(Per Cent) 


Component 


Standard of Excellence 
Both Major Components 
Problem Solving Only 
Subject Matter Only 


Acceptable Standard or Higher 
Both Major Components 
Problem Solving Only 
Subject Matter Only 


Table 3-7 
Grade 6 Mathematics: Booklets 1 and 2 
Percentage Distribution of Jurisdictions* Meeting 
Provincial Achievement Standards on the Total Test 
June 1991 


Jurisdictions 
Not Meeting 
Provincial 
Achievement 
Standard 
(Per Cent) 


Jurisdictions 
Meeting Prov- 
incial Achieve- 
ment Standard 
Reporting Category (Per Cent) 
otal Test: 


Standard of Excellence 


Acceptable Standard or Higher 


Jurisdictions 
Exceeding 


Jurisdictions 
Exceeding 
Provincial 
Achievement 
Standard 
(Per Cent) 


*Jurisdictions with fewer than six students are excluded because the statistical significance of the difference 
between the number actually meeting the standard and the number expected to meet the standard when 


calculated and reported is not educationally meaningful. 
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REPORTING CATEGORIES 


Table 3-8 shows the total marks possible and = jurisdiction and school reports. Variations in 
the provincial raw score results for the report- patterns of students’ responses to questions 
ing categories of Booklets 1 and 2 of the can help to indicate strengths and weak- 
Grade 6 Mathematics Achievement Test. nesses in local educational programs. 


It is important to stress that the averageson Statistics presented in this table are based on 


the various reporting categories cannot be results achieved by 29 768 students (those 
directly compared with one another. Rather, | who wrote the regular form and those who 
the results shown in Table 3-8 can best be wrote Form B). 


used in conjunction with parallel tables in the 


Table 3-8 
Grade 6 Mathematics: Booklets 1 and 2 
Raw Score Results by Reporting Category 
June 1991 . 


Maximum 
Possible Raw Score Raw Score 
Reporting Category Score Average Standard Deviation 


Total Test 


Major Component: 
Problem Solving 
Subject Matter 


Curricular Elements: 
Numeration 
Operations and Properties 
Measurement 
Geometry 
Graphing 
Problem-Solving Strategies 


Cognitive Level: 
Knowledge 
Comprehension 
Application/Problem Solving 
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PERCENTAGE OF STUDENTS CHOOSING EACH ALTERNATIVE 


Table 3-9 shows the percentage of students 
who chose each alternative (A, B, C, and D) 
for each multiple-choice question. The correct 
response (key) for each question is also identi- 
fied. The results shown in Table 3-9 can best 
be used in conjunction with the parallel tables 
in the jurisdiction and school reports. Varia- 
tions in patterns of students’ responses to 


questions can help to indicate strengths and 
weaknesses in local educational programs. 


Statistics presented in this table are based on 
results achieved by 29 768 students (those 
who wrote the regular form and those who 
wrote Form B). 


Table 3-9 
Grade 6 Mathematics: Booklets 1 and 2 
Provincial Results for Individual Multiple-Choice Questions* 
June 1991 


Distribution of 
Responses (%) 


Distribution of 
Responses (%) 


*The sum of the percentages for each question may be less than 100% because the No Response 
category is not included. This category is less than 1% for any one of these questions. 
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RESULTS FOR BOOKLET 3: NUMBER FACTS 


This section of the test measured student two minutes and the average number of 
performance in operations in addition, sub- questions students answered correctly in two 
traction, multiplication, division, and mixed minutes. 

operations; that is, it measured student speed 

and accuracy. Statistics presented in this table are based on 


results achieved by 31 930 students (those 
Figure 3-3 and Table 3-10 present the results who wrote the regular form and those who 


in terms of the average number of the 48 participated in the achievement-over-time 
questions students attempted to answer in study). 
Figure 3-3 


GRADE 6 MATHEMATICS 
RESULTS FOR NUMBER FACTS: SPEED AND ACCURACY TESTS 
June 1991 


Number of 


Questions 


Addition Subtraction Multiplication _ Division Mixed Operations 


The average number of questions 
answered correctly 


V7, The average number of 
Z questions attempted 


Table 3-10 
Grade 6 Mathematics 
Results for Number Facts: Speed and Accuracy Tests 
June 1991 


Time Allotted Number of | Average Number | Average Number 
Operations in Minutes Questions 


Addition 
Subtraction 
Multiplication 


Division 


Mixed Operations 
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GRADE 6 MATHEMATICS EXAMINER’S OBSERVATIONS 


The following is a discussion of specific areas of strength and of difficulty for Grade 6 students. 
Sample questions are provided to highlight the strengths and weaknesses of students achiev- 
ing at the acceptable standard and at the standard of excellence. For each sample question, 
the correct answer is marked with an asterisk. The percentage of students choosing each 


alternative is also provided. 


Acceptable Standard—Sample Questions and Commentary 


Use the chart below to answer 
question 21. 


21. What is the TOTAL enrolment of 
ALL the schools in the Eagleton 
School District? 


4.5 A. 1725 students 
88.8 *B. 1750 students 
3.1 C. 1775 students 
3.5 D. 1800 students 


Question 21 required students to read 

a simple chart and then apply a one-step 
problem-solving strategy to a real-life 
situation. 


12. Which group of symbols represents 
standard units of measure from 
SMALLEST to LARGEST? 


8.0 A. dm,cm,m 
4.6 B. hm, km, mm 
82.6*C. cm, m, km 
4.6 D. mm, km,m 


Question 12 required students to recognize 
units of measure and to apply a routine 
procedure. 


Generally, the Grade 6 teachers who analysed the 
standards built into the test felt that the test re- 
flected the nature of the Grade 6 Mathematics pro- 
gram. As well, these teachers felt that the provision 
of a section in which calculators were permitted was 
consistent with the NCTM Standards and with the 
philosophy and rationale of the role of technology in 
the Elementary Mathematics Curriculum Guide, 
1982. The teachers also felt that because the pro- 
gram has been in operation for almost 10 years, the 
test items were predictable; in other words, there 
were no surprises in the content of the test. 


General Statement on Student Performance 


Overall, students were able to solve one-step prob- 
lems, complete routine operations, and apply their 
knowledge to straightforward problem-solving ques- 
tions. As well, students showed a knowledge of and 
an ability to use basic measurement skills. 


Listed below are some areas where students per- 
forming at the acceptable standard and beyond 
showed a strong grasp of mathematics. 


Grade 6 students could do the following: 


* read simple charts and graphs (see question 21) 

* recognize place value, parallel lines, congruency, 
problem-solving stages, co-ordinates, ordered pairs, 
and units of measure and equivalencies (see 
question 12) 

¢ use routine procedures and operations in addition, 
subtraction, multiplication, division, and estima- 
tion (see question 46) 

* apply simple one-step problem-solving strategies to 
real-life situations (see question 21). 
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46. 9 )4321 equals However, students could not do the following well 


2.7 A. 318 RI enough: 
5.3 B. 329 R2 ; ; 
86.8*C. 480 R1 * complete multistep problems 
49 D. 491 R2 * transfer knowledge to new or real-life problem- 
Question 46 required students to perform solving situations ; ; 
a routine procedure in division. ¢ understand which operations to use when solving 
problems. 


Standard of Excellence—Sample Questions and Commentary 


Use the graph below to answer question 24. 11. 1m+4cm+2 mm equals 


ATTENDANCE AT THE CALGARY STAMPEDE 7.7 A. 10402 mm 
17.0 B. 1402 mm 
47.1*C. 1042 mm 
28.0 D. 142mm 


Question 11 required students to manipulate mixed units. 


Use the information below to answer question 53. 


A cross-country skier goes 3 kilometres in 15 minutes. 


24. The graph shows the attendance at 
the Calgary Stampede for one week. 
Saturday’s attendance was DOUBLE 
that of Sunday’s. Which circle should 
be filled in to show Saturday’s atten- 


dance? 

38 AZ 53. At this rate, how long will it take the skier to go 15 km? 
63.1 *B. Y 8.3 A. 150 min 
25.6 C. X 6.0 B. 90min 

74 D.V 61.6*C. 75 min 

23.9 D. 45 min 

Question 24 required students to read and ; ; : 
interpret a graph, and to apply a multistep Question 53 required students to apply ratios to a multistep 
problem-solving strategy to a real-life problem using a real-life situation. 


situation. 


Students who achieved at the standard of excellence demonstrated more success and under- 
standing in solving symbolic and multistep problems than other students did. Specifically, 
those students achieving at the standard of excellence could do the following: 


* demonstrate an understanding of equivalent fractions and place value in a problem-solving 
setting 

* recognize flips, slides, and cylinder construction 

¢ read and interpret graphs in problem-solving situations (see question 24) 
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* manipulate mixed units, arrange integers, calculate averages, add decimals, convert frac- 
tions to decimals, apply ratios, and find area, volume, and perimeter (see questions 11 and 
53) 

° apply more complex multistep problem-solving strategies to real-life situations (see 
questions 24 and 53). 


These students generally showed strengths in all areas that were assessed. 


New Features of the Test 


The introduction of a calculator and of the “cluster” questions on the June 1991 Grade 6 Math- 
ematics Achievement Test was well accepted by most teachers. Teachers remarked that the 
test could have been done without using a calculator; therefore, students without calculators 
were not disadvantaged. Teachers noted that more than adequate time was provided for stu- 
dents to complete Booklet 1 and Booklet 2. 


Issues 


Some teachers and students felt that not enough time was provided to complete the five timed 
tests. These tests are designed to test only speed and accuracy with number facts. To do this, 
a time element is built in to the tests to discriminate between students who know the number 
facts and those who do not. i 


Concluding Comments 


Overall, the results showed that not enough students performed at the acceptable standard 
and that a larger number of students than expected performed at the standard of excellence. 
In other words, although some students are achieving very well, most students are not learn- 
ing mathematical concepts and skills well enough. From a careful consideration of the ques- 
tions that students found difficult, it seems that students lack understanding in some of the 
concepts and their applications to real-life situations. Students need more work and time with 
manipulatives at the concrete level to help them understand abstract concepts. Further, an 
increased emphasis on problem solving in a variety of contexts would help students connect 
their newly acquired mathematical concepts to real-life situations. 
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SECTION 4 


GRADE 9 SOCIAL STUDIES 


GENERAL DESCRIPTION 


The Grade 9 Social Studies Achievement Test 
had two parts. Part A consisted of 60 
multiple-choice questions worth 70% of the 
total test mark. These questions assessed 
students’ knowledge as well as their 
understanding and application of process 
skills. 


Part B consisted of one written-response 
question worth 30% of the total mark. Papers 
were scored on two dimensions: Support of 
Position, worth 20%, and Expression, worth 
10%. 


The total writing time for the test was 1 3/4 
hours. The questions on the test were 
organized into two components: 


¢ Knowledge (facts, concepts, generalizations) 
¢ Process Skills 


Statistics for total test and for components 
were based on the results achieved by 27 646 
students (24 720 wrote the regular form, and 
2 926 participated in the achievement-over- 
time study, as shown in Table 4-3). This 
section of the report answers the following 
questions: 


¢ How many Grade 9 students wrote each 
form of the test or were absent and exempt? 

¢ What percentage of Grade 9 students 
attained the acceptable standard in Social 
Studies according to provincial criteria? 

¢ What percentage of Grade 9 students 
attained the standard of excellence in 
Social Studies according to provincial 
criteria? 

¢ What did Grade 9 students know and what 
could they do in Social Studies? 

¢ What parts of the Social Studies curriculum 
caused Grade 9 students difficulty? 


SUMMARY OF RESULTS 


REsuLts IN RELATION TO STANDARDS 


Results show that 67.9% of students who wrote the test achieved the acceptable standards on 
both components and 5.1% achieved the standards of excellence on both components. The 


results were lower than expected. 


AVERAGE SCORE 


The average total test score was 60.5% with a standard deviation of 15.2. 


CONTENT OF THE TEST 


The test was based on the 1989 Alberta Social 
Studies Program of Studies. The test 
measured two major components: 


¢ Knowledge (facts, concepts, generalizations) 
¢ Process Skills 


All questions were drawn from the content of 
the three topics prescribed for Grade 9: 


* Topic A: Economic Growth: U.S.A. 
* Topic B: Economic Growth: U.S.S.R. 
¢ Topic C: Canada: Responding to Change 


eEOn 


The multiple-choice section of the test covered 
all three topics and had 33 per cent emphasis 
on the knowledge component and 37 per cent 
emphasis on the process skills component. 


The written-response portion assessed 
students’ communication skills. Therefore, 
the total emphasis was on process skills. The 
assignment presented to students related to 
Topic C: Canada: Responding to Change. 


ACHIEVEMENT TEST BLUEPRINT 


The test blueprint for Part A: Multiple Choice shows the distribution of questions according to 
the curricular content area (topic) being assessed and according to the knowledge and process 
skill objectives called for by the question. 


Table 4-1 
Grade 9 Social Studies 
Achievement Test Blueprint 
Part A: Multiple Choice 
June 1991 


Topic/ 

Concept Topic A Topic B Topic Cc 

Reporting 

Categories | Economic Growth: U.S.A. Economic Growth: U.S.S.R. Canada: Responding To 
Change 


Objective Centrally 
Reporting | Industri-| Market | Quality | Geog- i-| Planned | Quality | Tech- | Mixed i 
Categories | alization | Economy | of Life | raphy | alization | Economy | of Life | nology | Economy | of Life 


Under- 1, 2,3, | 10,12,17) 13,15, 36,37, | 44,49, 43, 46, 
stands 19 38 50,51] 52 
general- 

izations, 

concepts, 

related 

concepts, 

terms, 

and 

facts 

Locating, 21 40 
interpret- 

ing, and 

organiz- 

ing 

Analysing, 5, 11, 18 26, 31 
synthesiz- 

ing, and 

evaluat- : 

ing 


of 


Total 
Score 


Process Skill 
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The test blueprint for Part B: Written Response shows the writing assignment presented to 
students and the emphasis of communication that was assessed. 


Table 4-2 


Grade 9 Social Studies 
Achievement Test Blueprint 
Part B: Written Response 


June 1991 


Proportion 
Writing Assignment Reporting Category of Score 


Recycling is a growing concern in Alberta. 


HOW SHOULD THE GOVERNMENT 
OF ALBERTA BE INVOLVED IN 
RECYCLING PROGRAMS? 


In presenting your opinion, consider economic 
and environmental factors. 


Your written response may take the form 
of a letter to the government, a speech to 
your class, a newspaper editorial, or a 
position paper. 


To support your opinion, use examples and 
evidence from the Information to Consider 
on page 25 of the test booklet, as well as from 
YOUR KNOWLEDGE OF ECONOMIC 
SYSTEMS. 


ee 


Support of Position: 20% 
Students are expected to 

support their position 

by organizing their points 

of argument convincingly, 

and by using specific and 

pertinent examples and 

details to illustrate their 

points of argument. 


Expression: 

Students are expected to 
express their ideas and 
arguments with clear, 
specific, accurate language, 
and to use correct conven- 
tions of spelling, punctuation, 
and grammar. 


STUDENT PARTICIPATION 


In June 1991, principals reported a total who were absent, and who were exempted. 
population of 32 553 students in Grade 9. Results for students in French Immersion or 
Table 4-3 presents the number and percent- Francophone programs are reported sepa- 
age distribution of students who wrote the rately to participating schools and jurisdic- 
Grade 9 Social Studies Achievement Test, tions. 


Table 4-3 
Grade 9 Social Studies 
Student Participation 
June 1991 


Number Percentage 
Category of Students of Students 


Other Students Who Wrote:*** 


Francophone Students—Wrote in French 
Francophone Students—Wrote in English 
French Immersion Students—Wrote in French 
French Immersion Students—Wrote in English 
Language-of-Testing Students—Wrote in French 
Language-of-Testing Students—Wrote in English 


Students Absent 


Students Exempted from Writing 


Categories of Exemption: 


1. Special Needs Students 

2. Subject Was Not Taught This Term 

3. ESL Students 

4. Language of Instruction Was Not English 
5. Other (as approved by the Superintendent) 


Total Principals’ Reported Population: 
Test Day, June 1991 


Grade 9 Enrolment: September 30, 1990 35 512 


* Provincial results are based on test scores achieved by students who were in the “regular” (Mathematics or Science 
or Social Studies) programs. Such students are defined as those who were instructed in English and wrote the 
English form of the achievement test, and those students who were instructed in a language other than English 
or French and wrote the English form of the test. These students are indicated by the shaded area of the table. 


** Regular Form refers to the unmodified 1991 test. The modified tests are described in Section 8 of this report. 


***Results for Francophone or French Immersion students or for students participating in the language-of-testing 
study are reported directly to participating schools and jurisdictions and are not calculated in the provincial 
results. 
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RESULTS IN RELATION TO STANDARDS 


Figures 4-1 and 4-2 and Table 4-4 show the percentage of students achieving the acceptable 
standards and the standards of excellence on components of the test. The levels of performance 
were lower than expected for the acceptable standards as well as for the standards of excel- 
lence. 


Figure 4-1 


GRADE 9 SOCIAL STUDIES 
PERCENTAGE OF STUDENTS ACHIEVING ACCEPTABLE 
STANDARDS OR HIGHER ON COMPONENTS OF THE TEST 
June 1991 


Knowledge Process Skills Both Componen 


Achievement Standards* 
(See Appendix A) 


*the percentage of students in the province expected to meet the acceptable standards or higher 
**the percentage of students in the province who met the acceptable standards or higher 


Actual Results** 


Figure 4-2 
GRADE 9 SOCIAL STUDIES 
PERCENTAGE OF STUDENTS ACHIEVING STANDARDS OF 
EXCELLENCE ON COMPONENTS OF THE TEST 
June 1991 


LLL 


Knowledge Process Skills | Both Components 


Achievement Standards* Actual Results** 


(See Appendix A) 


*the percentage of students in the province expected to meet the standards of excellence 
**the percentage of students in the province who met the standards of excellence 
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Table 4-4 
Grade 9 Social Studies 
Students Achieving Standards on Components of the Test 
June 1991 


Provincial | Provincial 


Possible 
Score 


Reporting Category 


Both Major Components*** | 33 + 67 29 + 53 10 2765 | 1402 
Facts, Concepts, 

Generalizations Only 33 29 15 4147 | 3081 
Process Skills Only 67 53 15 4147 | 2711 


Acceptable Standard 
or Higher 


Both Major Components*** 33 + 67 17+31 80 
Facts, Concepts, 
Generalizations Only 33 17 85 


Process Skills Only 85 


67 31 
Below Acceptable Standard N/A N/A N/A N/A 3 898 14.1 
on Both Components 


*The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is 
the lowest score students must achieve for their performance to be judged “acceptable” and/or “excellent” in 
relation to curricular expectations. See Appendix A. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the 
Provincial Assessment Standard. See Appendix A. 


***In previous years, a standard for the total test was reported. An individual could meet the standard for the test 
overall but might not achieve the standard for both components of the test. The new method of reporting allows 
for a more meaningful interpretation of the results. To allow comparison with results from previous years, the 
total test score is reported in Table 4-5. 
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Table 4-5 shows the percentage of students achieving the acceptable standard and the stand- 
ard of excellence on the total test. These levels of performance were lower than the expecta- 


tions for both standards. 
Table 4-5 
Grade 9 Social Studies 
Students Achieving Standards on the Total Test 
June 1991 


Possible 


Reporting Category Score 


Total Test: 


Standard of Excellence 


Acceptable Standard 
or Higher 


Provincial | Provincial 
Maximum | Assessment | Achievement 
Standard* 
(Raw Score) 


100 82 15 4147 2 242 8.1 
100 48 85 23 499 | 21 809 78.9 


Standard**} Expected| Actual | Actual 
(Per Cent) | Number | Number] Per Cent 


*The Provincial Assessment Standard is a score determined by appropriate standard-setting procedures and is the 
lowest score students must achieve for their performance to be judged “acceptable” and/or “excellent” in relation 


to curricular expectations. See Appendix A. 


**The Provincial Achievement Standard refers to the percentage of students expected to meet or exceed the Provin- 


cial Assessment Standards. See Appendix A. 


It should be noted that the actual percentages 
for Grade 9 Social Studies (8.1 and 78.9) are 
based on the 27 646 students in the regular 
programs who wrote the test. 


If, however, the percentages are based on the 
total population reported by principals 
(32 553), less the French Immersion and 
Francophone students who wrote the test 
(1 573), the percentages would be: 
7.2% achieving the standard of excellence 
70.4% achieving the acceptable standard 


If the percentages are based on the Septem- 
ber 30, 1990, Grade 9 enrolment (35 512), less 
the French Immersion and Francophone 
students who wrote the test (1 573), the 
percentages would be: | 
6.6% achieving the standard of excellence 
64.3% achieving the acceptable standard 


It is emphasized that the above percentages, 
based on total population and enrolment 
figures, present the worst possible scenario. 


- 39 - 


It is highly unlikely that none of the students 
who was absent, exempt, or not accounted for 
would have achieved standards. The absence 
of information on these students is nonethe- 
less problematic. 


The number of students achieving the accept- 
able standard and the standard of excellence 
for each jurisdiction was analysed to deter- 
mine whether jurisdictions were below, meet- 
ing, or exceeding provincial achievement 
standards. Jurisdictions classified as meeting 
provincial achievement standards were those’ 
for which the difference between the actual 
number of students and the expected number 
of students at or above standards was not 
statistically significant. A 95% confidence 
interval was used; this criterion means that 
differences are only reported when there is a 
5% or smaller probability that a difference of 
that size could occur by chance. The results 
are reported in Tables 4-6 and 4-7. The 
percentages in the tables are based on 200 
jurisdictions (including private schools). 


Table 4-6 


Grade 9 Social Studies 


Percentage Distribution of Jurisdictions* Meeting 
Provincial Achievement Standards on Components of the Test 


June 1991 
Jurisdictions 
Not Meeting 
Provincial 
Achievement 
Standard 
Component (Per Cent) 
Standard of Excellence 
Both Major Components 
Facts, Concepts, and Generalizations Only 
Process Skills Only 


Acceptable Standard or Higher 
Both Major Components 
Facts, Concepts, and Generalizations Only 
Process Skills Only 


Table 4-7 


Grade 9 Social Studies 


Jurisdictions 
Meeting Prov- 
incial Achieve- 
ment Standard 
(Per Cent) 


Percentage Distribution of Jurisdictions* Meeting 
Provincial Achievement Standards on the Total Test 


June 1991 


Jurisdictions 
Not Meeting 
Provincial 
Achievement 
Standard 


Reporting Category (Per Cent) 


Total Test: 


Standard of Excellence 


Acceptable Standard or Higher 


Jurisdictions 
Meeting Prov- 
incial Achieve- 
ment Standard 
(Per Cent) 


Jurisdictions 
Exceeding 
Provincial 
Achievement 
Standard 
(Per Cent) 


Jurisdictions 
Exceeding 
Provincial 
Achievement 
Standard 
(Per Cent) 


*Jurisdictions with fewer than six students are excluded, as the statistical significance of the difference between the 
number actually meeting the standard and the number expected to meet the standard when calculated and reported 


is not educationally meaningful. 
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RESULTS FOR PART A: MULTIPLE CHOICE 


REPORTING CATEGORIES 


Table 4-8 shows the total marks possible and 
the provincial raw score results for the report- 
ing categories of the multiple-choice portion of 
the Grade 9 Social Studies Achievement Test. 


It is important to stress that the averages on 
the various reporting categories cannot be 
directly compared with one another. Rather, 
the results shown in Table 4-8 can best be 
used in conjunction with parallel tables in the 


jurisdiction and school reports. Variations in 
patterns of students’ responses to questions 
can help to indicate strengths and weaknesses 
in local educational programs. 


Statistics presented in this table are based on 
results achieved by 25 709 students (those 
who wrote the regular form and those who 
wrote Form R). 


Table 4-8 
Grade 9 Social Studies 
Raw Score Results by Reporting Category 
Part A: Multiple Choice 
June 1991 


Reporting Category 


Topic A: Economic Growth: U.S.A. 
Topic B: Economic Growth: U.S.S.R. 
Topic C: Canada: Responding to Change 


Recall of Knowledge: Facts, Concepts, 


and Generalizations 
Application of Process Skills 


Recall of Knowledge: 
Topic A 
Topic B 
Topic C 


Application of Process Skills: 


Locating, Interpreting, and Organizing 
Analysing, Synthesizing, and Evaluating 
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Questions 


Number 
of Raw Score 


Average 


Raw Score 
Standard Deviation 


PERCENTAGE OF STUDENTS CHOOSING EACH ALTERNATIVE 


Table 4-9 shows the percentage of students 
who chose each alternative (A, B, C, and D) 
for each multiple-choice question. The correct 
response (key) for each question is also identi- 
fied. 


The results shown in Table 4-9 can best be 
used in conjunction with the parallel tables in 


the jurisdiction and school reports. Variations 
in patterns of students’ responses to questions 
can help to indicate strengths and weaknesses 
in local educational programs. 


Statistics presented in Table 4-9 are based on 
results achieved by 25 709 students (those 
who wrote the regular form and those who 
wrote Form R). 


Table 4-9 
Grade 9 Social Studies 
Provincial Results for Individual Multiple-Choice Questions* 
June 1991 


aiaaiaigay of Responses 
(f] 


Key B D 


1 
2 
3 
4 
5 
6 
7 
8 
9 


Pusteibution of Responses 
(9) 


Item Key A B C D 


*The sum of the percentages for each question may be less than 100% because the No Response 
category is not included. This category is less than 1% for any one of these questions. 
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RESULTS FOR PART B: WRITTEN RESPONSE 


Results for Part B: Written Response are most 
clearly understood in the context of the as- 
signment students responded to and in the 
context of the scoring guides. The average 
‘raw score for written response was 17.0 out of 


a possible 30, with a standard deviation of 5.2. 


Complete scoring guides are available from 


the Student Evaluation Branch, Alberta 
Education. 


All schools should have extra copies of the 
Part B: Written Response test to use in con- 
junction with information provided in this 
Provincial Report. 


ScORING RELIABILITY 


Although the papers were scored on a one- 
marker system, 192 randomly selected papers 
were re-marked so that a second set of scores 
was available to confirm scoring consistency. 
Of the scores awarded on the second reading, 
93.5% were identical to the original score on 
the same scale or varied by only one point. It 
is important to note that the one-marker 
system produces results that are reliable for 
groups of 25 or more students. Achievement 
_ test scores, however, are less reliable for 
individual students. 


The results presented in Table 4-10 are best 
considered in terms of the percentage of 


students that markers judged to have pre- 
sented work that was 3 (Acceptable) or higher 
for any reporting category. It is possible to 
draw conclusions about local program 
strengths and weaknesses by comparing local 
percentages of 3 (Acceptable) or higher scores 
on each reporting category with the provincial 
percentages. 


Statistics presented in Table 4-10 are based 
on results achieved by 25 709 students (those 
who wrote the regular form and those who 
wrote Form R). 


Table 4-10 
Grade 9 Social Studies 
Part B: Written Response 
Percentage Distribution of Written-Response 
Scale Points by Scoring Category 
June 1991 


Scale Point Support of Position 


5 (Excellent) 


4 (Good) 


3 (Acceptable) 


2 (Limited) 
1 (Poor) 


Ins (Insufficient or 
No Response) 
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GRADE 9 SOCIAL STUDIES EXAMINER’S OBSERVATIONS 


MULTIPLE CHOICE 


Students found the multiple-choice section of the test very challenging. The following is a 
discussion of specific areas of strength and of difficulty for Grade 9 students. Sample questions 
are provided to highlight the strengths and weaknesses of students achieving at the acceptable 
standard and at the standard of excellence. For each sample question, the correct answer is 
marked with an asterisk. The percentage of students choosing each alternative is also pro- 


vided. 


Acceptable Standard—Sample Questions and Commentary 


38. The most general definition of quality of life is 


5.9 A. your job satisfaction 
77.6 *B. how happy you are 

49 C. your level of education 
11.6 D. how much money you have 


Question 38 required students to have a basic understanding of quality of 
life. Students achieving at the acceptable standard have a good understanding 
of this concept. Few students confused quality of life with materialism. 


31. President Gorbachev’s attempted reforms MOST CLOSELY paralle) 


6.5 A. Lenin’s War Communism 
9.4 B. Stalin’s collectivization 
26.8 C, Stalin’s five-year plans 
57.1 *D. Lenin’s New Economic Policy 


Question 31 required students to apply their knowledge of historical events to 
current reforms in the Soviet Union. Results indicate that students achieving at the 
acceptable standard were unable to apply their knowledge of economic planning in 
this question; however, students achieving at the standard of excellence were clearly 
able to do so. 


The strengths of students who demonstrated an acceptable standard of performance include: 


° a basic, though limited, understanding of what a centrally planned economy—as in the 
Soviet Union before 1990—is, of the meaning of quality of life, of primary industry in an 
economy, and of aspects of one’s lifestyle (see question 38) 


¢ an ability to make simple inferences. 


Many students achieving at this standard had difficulty 


* recalling and applying some basic knowledge as outlined in the course of study (see 


question 31) 


* analysing, synthesizing, and evaluating sources of a more abstract nature. 


Standard of Excellence—Sample Questions and Commentary 


Use the information below to answer questions 7 and 9. 


Source I 


Nearly every immigrant wanted to work and this caused tension between 
American-borm workers and newcomers. Part of this conflict resulted from a 


lack of understanding of the causes of unemployment. Many American-bom 
workers did not understand that they were being displaced because the 
tremendous changes in manufacturing had caused the mechanization of their 
jobs. 


7. The MAIN point in source I is that 


16.9 A. immigration led to the mechanization of American industries 
4.0 B. workers tended to be more close-minded and racist than other people 
65.4 *C. workers lost their jobs to machines but blamed immigration 
13.6 D. owners hired more immigrants because they were willing to work for 
less 


Question 7 required students to identify the main point in source I. A fair 
percentage of students achieving at the acceptable standard were able to identify 
the main idea, whereas a high percentage of students achieving at the standard of 
excellence were able to do so. 


Source 


American-bom workers resented the immigrants, who were willing to work for 
less money. They grew angry because they felt their demands for higher wages 


and better working conditions were threatened by immigrants who were eager to 
work under any conditions, The competition between American-born workers 
and immigrants became intense as immigration increased and the worker 
population expanded one-third faster than the total population. 


9. Based on both sources, what generalization can be made about immigra- 
tion and economic growth? 


35.1 A. People move to areas where the economy is expanding. 
53.4 *B. Workers often believe that immigration creates unemployment. 
5.0 C. More skilled workers than unskilled workers tend to immigrate, 
6.3 D. Immigration has lowered profits and slowed the growth of factories. 


Question 9 required students to use a higher level cognitive process. They 
had to select the generalization that could be made based on the two sources. 
Many students achieving at the acceptable standard and some students 
achieving at the standard of excellence had problems selecting the correct 
generalization. Too many students in both groups were drawn to the first 
alternative. 
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aa 


Students achieving at the standard of excellence could 


¢ readily extract information from a reading passage and use it to support their main 


jdea 


* identify an author’s point of view in a reading passage (see question 7) 
¢ make judgments using their knowledge of economic growth 
¢ show a clear understanding of the effects of technology; for example, computer use in 


industry 


¢ show a good understanding of the economic, social, and environmental implications of 


technology on our way of life. 


Students achieving at this standard had difficulty 


* interpreting source documents that presented highly abstract ideas and synthesizing 


the information (see question 9) 


* recalling some basic facts and concepts. 


Assignment 
Recycling is a growing concem in Alberta. 


HOW SHOULD THE GOVERNMENT OF 
ALBERTA BE INVOLVED IN RECYCLING 
PROGRAMS? 


In presenting your opinion, consider economic and 
environmental factors. 


Your written response may take the form of a letter 
to the government, a speech to your class, a newspaper 
editorial, or a position paper. 


To support your opinion, use examples and evidence 
from the Information to Consider on page 25, as well 
as from YOUR KNOWLEDGE OF ECONOMIC 
SYSTEMS. 


Grade 9 Social Studies 
Percentage Distribution of Scores 
by Reporting Category for Written Response 


REPORTING CATEGORY 
SCORE SCALE 
POINTS Support of Position 


5 
(Excellent) 
4 
(Good) 

3 


(Acceptable) 


*the shaded portion indicates below-standard 
performance 


WRITTEN RESPONSE 


Although the quality of writing in social studies is 
better in 1991 than in 1987, not enough students are 
meeting the acceptable standard. Teachers were disap- 
pointed with many of the papers that they marked. The 
markers’ specific observations include the following: 


¢The majority of students had difficulty focusing on the 
issue. Writing was general rather than specific. 


¢ A number of students confused the levels of govern- 
ment. 


* Most students could state a position but had difficulty 
communicating ideas in support of their position. 


¢ Students wrote with conviction; their writing indicated 
a sincere concern for the environment. They believed 
that government should play an important role in 
solving recycling problems but that government is not 
fulfilling this function adequately. 


¢ A small number of clearly excellent papers were writ- 
ten. The quality of thought and expression in these 
papers far exceeded the expectations for Grade 9 Social 
Studies. 


The Student Evaluation Branch is preparing a booklet 
comprising samples of the full range of student writing, 
the standards used in marking, and a discussion of the 
relative strengths and weaknesses in student writing. 
Booklets will be sent to schools that have students en- 
rolled in Grade 9. 
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SECTION 5 


ACHIEVEMENT BY GENDER 


Information on the gender of students who Grade 6 Mathematics, and Grade 9 Social 

wrote the provincial achievement tests has Studies achievement tests in June 1991? 

been collected and data have been analysed 

and reported since 1989. ¢ Is the percentage of males and females 
meeting standards the same in each 

For the June 1991 administration, results are subject? 


presented in Figures 5-1 to 5-3. 
¢ What is the pattern of male and female 
This section of the report answers the follow- achievement across grades 3, 6, and 9? 


ing questions: 
e Are the 1991 results for individual sub- 


¢ What is the proportion of males and jects similar to or different from those of 
females who wrote Grade 3 Science, 1989 and 1990? 
Figure 5-1 


NUMBER OF ACHIEVEMENT TESTS WRITTEN BY GENDER* 
June 1991 


(thousands) 


Mathematics (Gr. 6) 
N = 33 821 N = 31 923 N = 27 588 


Male Female 


* Figures do not include tests where students did not identify their gender. The number of students who did not 
provide gender data is as follows: 
Grade 3 Science—-16 students 
Grade 6 Mathematics—7 students 
Grade 9 Social Studies—58 students 
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Figure 5-2 


PERCENTAGE OF STUDENTS ACHIEVING ACCEPTABLE STANDARDS 
OR HIGHER ON BOTH COMPONENTS OF THE TESTS BY GENDER 
June 1991 


Female 


*80% of students were expected to achieve the acceptable standards or higher on 
both components of the test. 


Figure 5-3 


PERCENTAGE OF STUDENTS ACHIEVING STANDARDS OF 
EXCELLENCE ON BOTH COMPONENTS OF THE TESTS BY GENDER 


Science (Gr. 3) Mathematics (Gr. 6) Social Studies (Gr. 9) 


| Male Female 


*10% of students were expected to achieve the standards of excellence on both 
components of the test. 
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OBSERVATIONS AND DISCUSSION 


NUMBER OF STUDENTS WRITING THE TESTS 


Results for 1991 reveal that more boys than 
girls wrote the achievement tests at each 
grade level. The pattern for grades 3 and 6 is 
supported by the 1989 and the 1990 data. For 
the Grade 9 data, the proportion of boys and 
girls changed over time. 


In 1989, the number of boys was marginally 
higher than the number of girls. Then in 


1990, the proportions reversed. This change 
-nd an explanation of the change in Grade 9 
requires further investigation to include 
absentee rates, enrolments, and age by gen- 
der. This information may be used to identify 
factors that may have contributed to the 
change. 


PERCENTAGE OF STUDENTS ACHIEVING STANDARDS 


Results for 1991 reveal that levels of achieve- 
ment are virtually the same for boys and girls 
for Grade 3 Science as well as for Grade 9 
Social Studies. For Grade 6 Mathematics, 
boys outperformed girls, particularly for the 
standard of excellence. The pattern for the 
Grade 6 Mathematics is supported by the 
1990 Grade 3 Mathematics results. 


The pattern for the 1991 Grade 3 Science 
results is not supported by the 1990 Grade 6 
Science results or the 1989 Grade 9 Science 
results, where boys outperformed girls, par- 
ticularly for the standard of excellence, in 
both years. 
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This change in pattern of achievement for 
boys and girls also exists in social studies, 
where there is virtually no difference in levels 
of achievement for boys and girls in the 1991 
Grade 9 Social Studies results but Grade 6 
girls outperformed boys in 1989. 


An explanation of these changes requires 
further investigation that will include more 
subjects and more administrations of tests of 
the same subjects. This will be possible when 
further data become available in 1992. 


SECTION 6 


ACHIEVEMENT BY AGE 

Information on the age of students who wrote ¢ What is the age distribution of students 
the provincial achievement tests was first who wrote the 1991 Achievement Tests? 
collected in June 1990. The 1990 data were 
analysed and reported in the 1990 Achieve- ¢ What relationship, if any, does age appear 
ment Testing Program Provincial Report. to have with achievement as measured by 
Similar information was collected for the June the 1991 provincial achievement tests? 
1991 adminstration. Results are presented in 
Table 6-1, and Figures 6-1 and 6-2. ¢ What changes in age distribution and 

achievement are evident between the 1991 
This section of the report answers the follow- and the 1990 test results? 


ing questions: 


Table 6-1 
Number of Achievement Tests Written by Age 
June 1991 


[sees nee! 
Grade 3 Science Grade 6 Mathematics pec Studies 


Rh 


Under 7 


Information 


33 837 | we | a 31930 | 100.0 ovo | 100.0 


*Age on the test day (June 11, 1991). 
The dashes in each column indicate that there were no students. 
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Figure 6-2 


ACHIEVEMENT BY AGE* 
PERCENTAGE OF STUDENTS ACHIEVING STANDARDS 
OF EXCELLENCE ON BOTH COMPONENTS OF THE TESTS 
June 1991 


Grade 3 Science 


ie 


Age of Students on June 11, 1991 


11 


Grade 6 Mathematics 


Age of Students on June 11, 1991 


Grade 9 Social Studies 


14 15 16 
Age of Students on June 11, 1991 


*Results for groups of fewer than 25 students are not shown on these graphs. 
**10% of students were expected to achieve the standards of excellence on both components of the test. 
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OBSERVATIONS AND DISCUSSION 


AGE GROUPS 


The 1991 data reveal that within each grade 
level, slightly over 90 per cent of students 
split into two age groups of approximately 
equal size: 


Grade 3: 8-year-olds and 9-year-olds 


Grade 6: 11-year olds and 12-year-olds 


Grade 9: 14-year olds and 15-year-olds 


The 1990 data exhibited the same patterns. 


STUDENT ACHIEVEMENT 


The 1991 data reveal that there is a high 
negative relationship between age and stu- 
dent achievement. This means that within a 
grade level, younger students’ achievement is 
higher than older students’ achievement. 
This is true for both the acceptable standards 
and the standards of excellence across all 
three grades. The same patterns and rela- 


tionships appeared in the 1990 results as well. 


This lower achievement of older students in 
the grade levels tested may be attributed to 
the interaction of several factors. The most 
likely factors are speculated to be repeating 
students, late entrants into formal schooling, 
and transferees from other provinces or 
countries where there may be curricular 
differences and/or the language of instruction 
is not their first language. 


Schools and jurisdictions whose results reflect 
negative relationships similar to the ones 
exhibited by the provincial results are encour- 
aged to identify the specific factors operating 
within their own classrooms and schools. 
Accurate identification of such factors would 
be the first step in the development of plans to 
reduce or eliminate this unwelcome relation- 
ship. 


Student Evaluation Branch welcomes obser- 
vations from school administrators on the 
factors they identify as being the major con- 
tributors to the prevailing negative relation- 
ships between age and achievement. 
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SECTION 7 


ACHIEVEMENT BY GRADE LEVEL 


Section 7 is an exploratory approach to re- 
porting results from the 1991 achievement 
tests. 


This section attempts to provide answers to 
the following question: 


What percentage of grades 3, 6, and 9 
Alberta students 


emay be achieving beyond grade level 
eare achieving at grade level 

may be achieving at grade level 

*are not yet achieving at grade level 


as measured by the June 1991 achieve- 
ment tests? 


To facilitate implementation of a results- 
based curriculum, an attempt is made in this 
section to analyse and report student achieve- 
ment by "levels". 


Within each curriculum, specific outcomes, 
expressed as the knowledge and skills to be 
acquired and developed, are arranged into 
sequences reflecting the developmental na- 
ture of learning and expectations for a learn- 
er’s increasing competence or confidence in 
the subject area. 


As students differ from each other in their 
rates of learning or development along the 
continuum, the levels of learning can be 
considered to be independent of age or the 
amount of time spent in school. 


Additionally, since all students do not neces- 
sarily achieve in chronological sequences but 
learn in different ways, the flexibility of a 
levels approach allows teachers to focus at all 
times on the individual needs of students. 


In summary, a results-based curriculum is 
vitally important to the educational process 
as it allows for and encourages progression 
while also accommodating individual differ- 
ences. 
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The 1991 achievement tests were designed to 
measure student achievement within the 
given grade level. However, there were a 
number of questions on the achievement tests 
that could be considered to measure more 
advanced levels of performance. For reporting 
purposes only, we have classified students 
within the levels framework, based on the 
following four definitions: 


1. May Be Achieving Beyond Grade Level 


Students who may be achieving beyond 
grade level are those students who achieved 
the standards of excellence for both major 
components of the test and correctly an- 
swered certain selected questions that 
assess more advanced levels of performance. 


2. Achieving At Grade Level 


Students who are achieving at grade level 
are those students who achieved the accept- 
able standards for both major components 
of the test and those students who achieved 
the standards of excellence but did not 
answer correctly the selected questions that 
assess more advanced levels of performance. 


3. May Be Achieving At Grade Level 


Students who may be achieving at grade 
level are those students who met the accept- 
able standard for one of the major compo- 
nents of the test but not for both compo- 
nents. 


4. Not Yet Achieving At Grade Level 


Students who are not yet achieving at grade 
level are those students who did not meet 
the acceptable standard for either major 
component of the test. These students are 
not yet considered to be performing at grade 
level. 


Tables 7-1, 7-2, and 7-3 present results from this new analysis for grades 3, 6, and 9 respec- 


ively: Table 7-1 
Grade 3 Science Achievement Test 
Distribution by Grade Level Achievement 
June 1991 


Students Who May Be Achieving Beyond Grade 3 Level 
Students Who Are Achieving At Grade 3 Level 
Students Who May Be Achieving At Grade 3 Level 
Students Who Are Not Yet Achieving At Grade 3 Level 


Table 7-2 
Grade 6 Mathematics Achievement Test 
Distribution by Grade Level Achievement 
June 1991 


Students Who May Be Achieving Beyond Grade 6 Level 
Students Who Are Achieving At Grade 6 Level 


Students Who May Be Achieving At Grade 6 Level 


Students Who Are Not Yet Achieving At Grade 6 Level 


Table 7-3 
Grade 9 Social Studies Achievement Test 
Distribution by Grade Level Achievement 
June 1991 


Students Who May Be Achieving Beyond Grade 9 Level 
Students Who Are Achieving At Grade 9 Level 


Students Who May Be Achieving At Grade 9 Level 


Students Who Are Not Yet Achieving At Grade 9 Level 


These results quantify the wide range of student achievement within a nominal grade level. 
The importance of this experimental analysis is that, for the first time, it shows performance in 
relationship to provincial expectations based on a “levels” curricula approach. 
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SECTION 8 


ACHIEVEMENT OVER TIME 


An important goal of Alberta Education is to 
measure and report changes in student 
achievement over time. Since 1983, the 
Student Evaluation Branch has been measur- 
ing the achievement of students in grades 3, 6, 
and 9 in the subject areas of language arts, 
mathematics, science, and social studies. The 
administration of the Achievement Testing 
Program follows a four-year cycle for each 
subject within each grade level. As a result, 
in 1983, 1987, and 1991, science was tested in 
Grade 3, mathematics was tested in Grade 6, 
and social studies was tested in Grade 9. This 
section of the report answers the question 


¢ Has achievement as measured by individual 
provincial achievement tests in Grade 3 
Science, Grade 6 Mathematics, and Grade 9 
Social Studies changed since 1983 and 1987? 


Two studies were carried out. The first study 


was designed to compare achievement since 
1983 on the multiple-choice questions in 
Grade 3 Science, Grade 6 Mathematics, and 
Grade 9 Social Studies. The second study was 
designed to compare achievement since 1987 
on the written-response part of the Grade 9 
Social Studies test. The results of these 
special studies follow. 


ACHIEVEMENT OVER TIME: MULTIPLE CHOICE 


Stupy DEsIGN 


In the past, achievement-over-time studies 
carried out by the Student Evaluation Branch 
involved the re-administration of earlier tests 
to a sample of students who were also writing 
the current tests. Because these students had 
to write two tests, the re-administration was 
either one week before or one week after the 
current test administration. However, moti- 
vational levels and state of preparation were 
likely different from when the tests were 
written during the original administrations. 
As well, the additional testing time required 
of students participating in the study was a 
concern. 


To address these issues, a new design was 
implemented for the 1991 achievement-over- 
time study. Random samples of students 
wrote all or part of previous achievement tests 
under the same conditions as the earlier test 
administrations. That is, the achievement- 
over-time tests were administered to the 
random samples at the same time students in 
the rest of the province were writing the 
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current achievement test. To maintain the 
validity of the tests for the 1991 students, the 
achievement-over-time tests were developed 
to match the blueprints of the 1991 tests by 
selecting questions from the previous tests 
that were still appropriate in 1991. Questions 
that were no longer appropriate, because of 
curricular changes, were replaced with 1991 
questions. As a result, it was possible to use 
the 1983 and 1987 Grade 3 Science tests in 
their entirety, but only parts of the Grade 6 
Mathematics and Grade 9 Social Studies tests 
were still appropriate. As well, a subset of 
students who were randomly selected to 
participate in the achievement-over-time 
study wrote the current 1991 tests so that we 
could verify that the samples in the study 
were representative of the province. 


Table 8-1 shows the composition of the 
achievement-over-time (AOT) tests in terms of 
the number of questions they contain from the 
1983, 1987, and 1991 tests. 


Table 8-1 
Comparison of the Achievement-Over-Time Tests 
by the Number of Questions from the Previous 
and Current Tests 


Grade 3 Science 
Number of 1983 Questions 
Number of 1987 Questions 
Number of 1991 Questions 


Grade 6 Mathematics 
Number of 1983 Questions 
Number of 1987 Questions 
Number of 1991 Questions 

Grade 9 Social Studies 
Number of 1983 Questions 
Number of 1987 Questions 
Number of 1991 Questions 


1983 
AOT Test 


*One question on the Grade 9 Social Studies 1987 AOT test was dropped because of a typographical 


error. 


METHODOLOGY 


Sampling of the schools involved in the study 
was carried out independently in grades 3, 6, 
and 9. Because of project design considera- 
tions, only schools with 24 or more students in 
regular English programs* who wrote 
achievement tests in 1990 in the grade being 
sampled were included in the groups from 
which samples were drawn. At each of the 
three grade levels, all eligible schools were 
identified and listed. From each of these lists, 
random samples of schools were selected. The 
sample sizes were targeted for approximately 
3 000 students at each grade level. Since the 
sampling was carried out independently at 
each grade level, some schools were randomly 
selected to participate at more than one 
grade. 


*See the first footnote on page 9. 


Before test materials were shipped, the par- 
ticipating schools submitted to the Student 
Evaluation Branch copies of their class lists. 
Within each class, students were randomly 
assigned to write either the AOT 1983 test, 
the AOT 1987 test, or the AOT 1991 test. 
Along with test materials, the class lists were 
sent back to the schools with instructions 
about which test each student was to write. 


The students who wrote the 1983 achieve- 
ment-over-time test are referred to hereafter 
as the AOT83 group, the students who wrote 
the 1987 achievement-over-time test are 
referred to as the AOT87 group, and the 
students who wrote the 1991 test are called 
the AOT91 group to differentiate them from 
the students in the rest of the province who 
wrote the 1991 regular achievement test. 
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RESULTS 


Before comparisons were made between the 
AOTS83 group and the students writing in 
1983, and the AOT87 group and the students 
writing in 1987, it was necessary to verify 


that the students selected to participate in the 


special study were representative of students 


throughout the province. Since approximately 


one-third of the students sampled were 


assigned to the AOT91 group, it was possible 
to compare their results with the provincial 
results: Because the large sample sizes 
allowed us to detect very small differences, 
only differences where the probability was 
less than 0.01 were considered to be signifi- 
cant. Table 8-2 presents the findings. 


Table 8-2 
Comparison of Results for the AOT91 Groups 
with Provincial Results 


Grade 3 Science 
Raw Score Mean 
Standard Deviation 
Number of Students 
Grade 6 Mathematics 


Raw Score Mean 
Standard Deviation 
Number of Students 
Grade 9 Social Studies 
Raw Score Mean 
Standard Deviation 
Number of Students 


Table 8-2 shows that at all three grades, the 


means of the AOT91 group and of the province 


were within one raw score point. None of 
these differences was statistically significant 
at the 0.01 level. As there were no significant 
differences between the means of the AOT91 
groups and the provincial means, we con- 
cluded that the samples of students in the 
special study were representative of all regu- 
lar students in the province. 
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Because Grade 6 Mathematics and Grade 9 
Social Studies students in the special study 
responded to a number of 1991 questions, it 
was possible to carry out an additional check 
that the random assignment resulted in 
groups that were representative of the prov- 
ince. Table 8-3 presents the results of these 
comparisons. 


Table 8-3 


Comparison of AOT Samples with Provincial Results 


AOT83 AOT87 
Group Province Group Province 


Grade 6 Mathematics 
No. of 1991 Questions 
Raw Score Mean 
Standard Deviation 
Number of Students 

Grade 9 Social Studies 
No. of 1991 Questions 
Raw Score Mean 
Standard Deviation 
Number of Students 


On the 1991 questions, the differences be- 
tween the means of the students in the special 
study and the means of students in the rest of 
the province are within 0.2 raw score points, 
as can be seen from Table 8-3. These differ- 
ences are not statistically significant at the 
0.01 level. Based on these analyses, we 
concluded that the random assignment was 
successful and that the samples are repre- 
sentative of the province. 


Since it was established that our samples 
were representative of the province, we were 
able to proceed with the comparisons between 
the 1991 students in the study and the stu- 
dents in 1983 and 1987. As stated earlier, in 
the methodology section, we restricted our 
sampling to schools with an estimated mini- 
mum size of 24 students in the grade being 
sampled. When making comparisons with the 
previous years, we attempted to exclude 


students who wrote in the earlier years who 
were from schools that were smaller than the 
smallest school size in the 1991 samples. 
However, individual student data were not 
available for the 1983 Grade 3 Science and 
Grade 6 Mathematics tests. It was not possi- 
ble, therefore, to exclude from these compari- 
sons students from schools smaller than the 
school sizes in the 1991 sample. 


In the comparisons, data from all 1983 Grade 
3 Science and Grade 6 Mathematics tests 
were used. 


Table 8-4 shows, for each grade, the number 
of common questions that were answered by 
both the 1991 students and the students in 
the previous years, the means and standard 
deviations for these questions, and the 
number of students on which these statistics 
were based. 


- 60 - 


Table 8-4 
Achievement Over Time 
Comparison of Descriptive Statistics 


Grade 3 Science 
Number of Questions 
Raw Score Mean 
Standard Deviation 
Number of Students 

Grade 6 Mathematics 
Number of Questions 
Raw Score Mean 
Standard Deviation 
Number of Students 

Grade 9 Social Studies 

Number of Questions 

Raw Score Mean 

Standard Deviation 

Number of Students 


Regular 
1987 
Students 


Regular 
1983 
Students 


*The mean for the 1991 students in the AOT group is significantly different from the mean of the previous admin- 


istration at the 0.001 level. 


**The original student response data for 1983 were not available. Question difficulties were used to reconstruct the 
means, but standard deviations could not be reconstructed. 


Table 8-4 shows that for all but one com- 
parison of interest, the 1991 students scored 
significantly higher than the students in 
1983 and 1987. The exception is Grade 9 
Social Studies, where the achievement in 
1991 is the same as the achievement in 
1987. Specifically, Grade 3 Science students 
in 1991 achieved between 3.1 and 3.9 raw 
score points higher than the 1983 popula- 
tion based upon a 99% confidence interval. 
This means that the chances are 99 in 100 
that the difference between all 1991 stu- 
dents in regular English Grade 3 Science 
programs and the 1983 students falls within 
the range specified. Similarily, Grade 3 
Science students in 1991 achieved between 
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3.3 and 4.6 raw score points higher than the 
1987 population. Grade 6 Mathematics stu- 
dents achieved between 1.8 and 2.8 raw score 
points higher in 1991 than the 1983 popula- 
tion, and between 0.8 and 1.8 raw score points 
higher in 1991 than the 1987 population. 
Grade 9 Social Studies students achieved 
between 0.2 and 0.9 raw score points higher in 
1991 than the 1983 population, but their 
achievement was not significantly different 
from the 1987 population. 


More detailed analyses of the changes in 
achievement are described below for each 
grade. 


GRADE 3 SCIENCE 


A comparison of the results of the 1983 test 
written in 1983 and again in 1991 indicates 
that more students in 1991 answered ques- 
tions correctly than students did in 1983. 
Although higher percentages of students 
achieved greater success in both process skills 
and subject matter, the strength of the 1991 
Grade 3 Science students was unquestionably 
in the area of process skills. 


In 1991, the increase in the percentage of 
students correctly answering 1983 process 
skills questions was dramatic: 


¢ Approximately 30% more students could 
accurately collect data using their obser- 
vation skills. They also knew the differ- 
ence between observations and infer- 
ences. 


¢ Approximately 15% more students could 
successfully group unfamiliar objects 
according to physical characteristics, 
using their knowledge and understand- 
ing of classification. 


¢ Approximately 10% more students could 
recognize correct graphical representa- 
tions of data and infer correct interpreta- 
tions from the graphed data. 


* Approximately 13% more students knew 
how to measure the length and volume of 
objects accurately. 


Students in 1991 also showed marked im- 
provement over 1983 students in their knowl- 
edge and understanding of science concepts: 


¢ Approximately 25% more students 
understood the capacity of objects made 
of different materials to float or sink in 
water. 


Approximately 18% more students 
understood that some reactions can be 
reversed and others cannot. 


¢ Approximately 26% more students knew 
the difference between materials that 


naturally decompose and those that do not. 


A comparison of the results of the 1987 test 
written in 1987 and again in 1991 indicates 
that student achievement has improved. An 
examination of the question-by-question 
results revealed that although achievement 
remained the same on some questions, on 
others a much larger percentage of 1991 
students answered correctly compared with 
1987 students. Students’ ability in the area 
of process skills has improved since 1987: 


e Approximately 15% more students could 
use their senses to accurately collect data 
based on observation. 


e Approximately 10% more students could 
measure volume accurately, rank objects 
based on comparative measurements of 
mass, and read temperatures on a ther- 
mometer. 


Approximately 10% more students had 
the ability to make interpretations from 
graphs and infer beyond the data pro- 
vided. 


e Approximately 15% more students could 
classify objects into groups based on 
physical characteristics. 


Although the greatest improvement was in 
the area of process skills, students also im- 
proved in the area of subject matter. Approxi- 
mately 10% more students had an under- 
standing of: 


e the differences between living and 
nonliving things. 


e shadows and how they are affected by 
the apparent shift of the sun’s position 
relative to the Earth’s rotation. 


° how to save energy in the home. 


¢ what the process of dissolving means. 
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One possible reason for the dramatic improve- 
ment in achievement compared with 1983 and 
1987 is that teachers are doing a better job of 
delivering science programs across Alberta, 
particularly in the teaching of process skills. 
Another reason is that teachers and students 
have greater access to resources and support 
that are imperative for an effective science 


GRADE 6 MATHEMATICS 


A greater increase in student achievement 
occurred from 1983 to 1991 than from 1987 to 
1991 in Grade 6 Mathematics. Specifically, 
increases over 1983 were evident in the fol- 
lowing strands and objectives: 


¢ Numeration—understanding of numbers, 
fractions, and percentages. 


¢ Measurement—recognition of time on the 
24-hour clock, knowledge of metric 
lengths, and application of perimeter and 
area. 


* Geometry—knowledge of terminology 
and recognition of a segment and a slide. 


¢ Graphing—application of knowledge to 
solve questions using a bar graph. 


By strand, most increases in achievement 
were in Numeration and Measurement. No 
important changes were found among ques- 
tions measuring the strands Operations and 
Properties, and Problem Solving. 


Under the cognitive level taxonomy, there was 
an overall increase betweeen 1983 and 1991. 
Large increases were noted on single ques- 
tions in the areas of Comprehension and 
Problem Solving, and large increases were 
noted on several questions in the areas of 
Knowledge and Application. 
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program. Still another reason is the current 
overlap of process skills in science with other 
subject areas, such as social studies and 
mathematics. This could account for the 
greater increase in process skills compared 
with subject matter. 


Interestingly, there was no achievement 
increase in routine procedural questions. 
Questions requiring quick routine calculations 
were answered correctly by roughly the same 
or a slightly smaller proportion of students in 
1991 as in 1983. This was evident in the non- 
calculator section. 


Increases in achievement since 1987 were 
generally not large. Large increases were 
evident in only three questions from the 
following strands: 


¢ Measurement—approximately 10% more 
students were able to solve a problem 
related to finding the volume of a box. 


¢ Graphing—approximately 10% more 
students were able to recognize similar 
shapes in a slide image. 


¢ Problem Solving—approximately 15% 
more students were able to solve a prob- 
lem related to the understanding of 
number patterns. 


Similarly, the achievement increase from 
1987 to 1991 with regard to cognitive levels 
was small but general. The three questions 
referred to above were in the areas of Know}l- 
edge (1) and Problem Solving (2). 


In summary, for Grade 6 Mathematics, our 
analysis seems to indicate that there was a 
general but small improvement in all strands, 
with the least improvement in the area of 
Operations and Properties. More emphasis 
seems to have been placed by teachers and 
administrators on understanding and the 
application of skills and knowledge, and less 
emphasis on routine operations without 
applications. Some causes for the greater 
increase over 1983 than over 1987 may be 
that the current curriculum was introduced 
shortly before the 1983 test, whereas it had 
been in operation for five years by the time of 
the 1987 tests. Teachers and education 
leaders, and therefore students, were 


GRADE 9 SociAL STUDIES 


A slight overall increase in student achieve- 
ment was found from 1983 to 1991. An ex- 
amination of the responses to the individual 
1983 questions revealed that only three 
questions showed large increases (10% or 
greater) in the percentage of students answer- 
ing correctly compared with 1983. These three 
questions were based on the knowledge objec- 
tives. 


The reason for the achievement increase in 
two of the questions could be that students in 
1991 have a stronger vocabulary than stu- 
dents did in 1983. The key words in these 
questions were “scarcity” and “bias,” and 
students needed to know the meaning of these 
terms to answer the questions correctly. The 
other question that showed improvement 
dealt with the Soviet Union. The 


increasingly aware of what was expected. 


It is interesting that in the basic facts portion 
of the test, there was a noticeably consistent, 
albeit small, decline from 1987 to 1991 (1983 
is not comparable). In addition, subtraction, 
multiplication, division, and mixed operations, 
the average number of questions attempted 
and the average number answered correctly 
declined slightly in 1991. We note that the 
questions were not identical for 1987 and 
1991 but were of similar type and expected 
difficulty. The results may be related to the 
reduced emphasis on routine operations 
without applications. 


achievement increase might, in part, have 
resulted from the intense media coverage of 
the events of the past year in the Soviet 
Union. Interestingly, one question showed a 
decrease in achievement of approximately 
14% between 1983 and 1991. Two of the 
response options to this question contained 
the word “materialism.” It is possible that 
with the shift in the focus of the curriculum 
from “materialism” to “quality of life,” 1991 
students found this question to be more 
difficult because they were not familiar with 
the word. 


No large differences in student responses 
were found between 1987 and 1991. 


CONCLUSIONS 


Ideally, when comparing achievement over 
time the same test is given over different time 
periods to samples of students who have not 
gained access to the test ahead of time. Since 
achievement tests are released after adminis- 
tration, it is not possible to guarantee that 
students writing in subsequent administra- 
tions have not seen the test. When developing 
the new study design, consideration was 


given to asking teachers whose students were 
participating in the study not to familiarize 
students with the previous achievement tests. 
However, we decided that would not be fair to 
the students involved, as it can be useful for 
students to know the types of questions they 
will be asked to answer. We realized at the 
time that this was a limitation with the design. 
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A telephone survey of a sample of teachers 
whose classes participated in the study re- 
vealed that the overwhelming majority of 
teachers did not have the 1983 tests to prac- 
tice with because the tests did not remain in 
schools after the 1983 administration. The 
few teachers who stated that they did have a 
copy of the 1983 test indicated that the 1983 
test was not given much attention. However, 
almost one-half of the teachers surveyed 
indicated that they had gone through all or 
part of the 1987 test with their students to 


provide them with an understanding of the 
format of the test. Very few teachers indi- 
cated that they used the 1987 achievement 
test as a practice test. Since for the grade 6 
and 9 achievement-over-time tests selected 
items from the previous tests were embedded 
with 1991 items, any effect of practice should 
have been minimized. Regardless, the 1987 
comparisons, especially for Grade 3 Science, 
should be interpreted with caution. We are 
confident that the 1983 comparisons are valid. 


ACHIEVEMENT OVER TIME: WRITTEN RESPONSE 


StTupy DESIGN 


Students’ writing performance has been of 
particular interest to educators and to the 
public since the beginning of the Achievement 
Testing Program. As a consequence of inter- 
est in how well students write, a study was 
initiated to compare 1987 writing with 1991 
writing in Grade 9 Social Studies. This was a 
qualitative or descriptive study that required 
the teacher-readers to take a research or 
"reader-as-observer" stance toward the papers 
they read, rather than the usual evaluative or 
"reader-as-assessor" stance of the teacher- 
marker. No attempt was made to rescore 
papers; rather, teacher-readers described 
features of 1987 and 1991 writing in two 
general categories: 


Defence of Position 
—development of ideas 
—persuasiveness 
—accuracy of content 
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Use of Language 
—vocabulary choices 
—control of conventions: spelling, 
punctuation, grammar 
—organization and structure 


In 1987, the written-response assignment 
involved short answers and one extended 
response. In 1991, the assignment was lim- 
ited to one extended response. In order to 
facilitate comparison of similar assignments, 
only the extended response from 1987 was 
compared with the 1991 assignment. 


Comparisons were made at two standards: 
Acceptable (3), which represents work at an 
acceptable level for students completing 
Grade 9, and Excellent (5), which represents 
outstanding work for students completing 
Grade 9. 


METHODOLOGY 


Papers read in the study were selected at Working alone and in pairs, and then discuss- 
random from June 1987 papers that received ing papers as a group, the teacher-readers 
scores of Acceptable (3) or Excellent (5) and described the papers for features of defence of 
from June 1991 papers that received scores of position and use of language. 
Acceptable (3) or Excellent (5). 

The teacher-readers then compared their 
A group of ten experienced teacher-readers, descriptions of 1991 and 1987 Acceptable (3) 
representing all major regions of the province and Excellent (5) papers and drew conclusions. 
and a variety of school settings, reviewed the 


selected papers. 
RESULTS 

The results of the study are given in tables work described at each level met the standard 
8-5 and 8-6. Each comment describes the established for the appropriate marking 
features of what students writing at a par- session. Teacher-readers agreed that papers 
ticular level did in 1991 compared with judged to be Acceptable (3) were considerably 
students in 1987. The wording in the tables better in 1991 than in 1987. As well, a higher 
comes directly from teacher-reader’s descrip- percentage of students were able to achieve 
tions. the Acceptable (3) level in 1991 than in 1987. 

Improvements were noted as well in the 
The level of expectations embedded in the papers scored Excellent (5); however, fewer 
1991 scoring criteria was increased signifi- students achieved this level in 1991 than in 


cantly from the 1987 criteria. However, the 1987. 


- 66 - 


Table 8-5 
Acceptable (3) Papers in Social Studies 9 


KEY FEATURES OF 1991 
ACCEPTABLE (3) PAPERS 


Defence of Position 


° could identify with issue on a personal 
level, focused 


* genuine concern, commitment toward topic 


° use of own experiences to support state 
ments 


© use of prior/media/other subject knowledge 
common 


* tendency to "broaden" topic to include other 
environmental concerns 


* insight shown but not consistent 
¢ sense of urgency regarding topic 


* simplification of concepts common 


° ideas generally organized 


KEY FEATURES OF 1987 
ACCEPTABLE (3) PAPERS 


Defence of Position 


° did not relate to topic, sense of remoteness 
from topic 


° lack of understanding of issue 


¢ limited defence of position, support was 
superficial 


¢ did not incorporate information from outside 
social studies classroom 


° identified with one of the speakers, not with 
a position 


¢ lacking in insight 
* emotionalism rather than conviction 


° basic social studies concepts generally 
understood but in a concrete manner only 


° ideas poorly organized 


* confusion over government levels and roles | Note: the question did not require students to 


Use of Language 


¢ paragraphing skills evident 


¢ specific words, dynamic vocabulary at- 
tempted 


* general attempt to use and spell social 
studies terms correctly 


* environmental "buzz words" and jargon 
common 
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address levels and roles of government 


Use of Language 
® poor organization and structure 
¢ limited vocabulary 
¢ frequent errors in spelling, punctuation, and 
grammar 


¢ "Motherhood" statements common 


Table 8-6 
Excellent (5) Papers in Social Studies 9 


KEY FEATURES OF 1991 
EXCELLENT (5) PAPERS 


KEY FEATURES OF 1987 
EXCELLENT (5) PAPERS 


Defence of Position Defence of Position 


¢ encouraged personal involvement/identified 
with topic 


° insightful, creative thought exhibited 


¢ focused, controlled, does not wander 


* social studies concepts incorporated 

® convincing, persuasive arguments 

¢ qualitative evaluation, used data well 
* more than one idea presented 


¢ clear delineation between roles of govern- 
ment/business/public 


* genuine concern with topic, healthy 
skepticism exhibited 


* occasional meaningless regurgitation of 
statistics to support argument 


¢ understood government levels, roles, limita- 
tions 
* good understanding of economic systems 


Use of Language 


° experimental use of vocabulary 


¢ sense of audience, with careful attention to 
spelling and conventions 


¢ variety in forms of expression 


¢ dramatic language used but generally well 
controlled 


¢ attitude of detachment from topic 


* little personal insight 


* no creativity in approach to topic or possible 
solutions 


e limited inclusion of social studies concepts 

* mechanical argumentation 

® position supported with multiple arguments 
° examples very specific 


° good use of data from outside social studies 
classroom 


° both sides of question understood 

° acknowledgment of flaws in argument or of 
the opposite view, and rebuttal 

Note: the question did not require students to 

address government levels, roles, limitations 

nor did it make reference to economic systems 

Use of Language 
° advanced technical vocabulary 


° good spelling and grammar 


* logical organization and good structure 


* accurate but pedantic language used 


CONCLUSIONS 


The question that guided the study—Has 
writing by Grade 9 Social Studies stu- 
dents improved from 1987 to 1991?—was 
answered in the affirmative. The 1991 stu- 
dents at the Acceptable (3) level of perform- 
ance produced better writing than did their 
1987 counterparts. Although a lower percent- 
age of students were able to achieve the 
Excellent (5) level of performance in 1991, 
those who did so produced better writing. 


A change in the scoring criteria between 1987 
and 1991 reflected an increase in standards 
and in the level of expectations for Grade 9 
Social Studies students. This may have 
contributed to the lower percentage of stu- 
dents able to achieve a score of Excellent (5). 
However, even with this increase in standards 
and criteria, a significantly greater number of 
students were able to achieve a score of Ac- 
ceptable (3). One possible conclusion is that 
while top-range students’ writing has not 
changed markedly since 1987, average-range 
students’ writing is better. 


A greater variety of response formats may 
also have contributed to the improved writing 
on the 1991 test. Students in 1991 could 
choose to write a letter, speech, editorial, or 
position paper, whereas 1987 students were 
limited to a prescribed letter format. 


In 1991, students addressed only one written 
response, whereas in 1987 students answered 
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four short written-response questions before 
attempting the extended response. This may 
have contributed to the higher number of INS 
(insufficient) responses in 1987. Within the 
confines of a test situation, some students 
may have experienced oversaturation with the 
topic before attempting the extended re- 
sponse. 


Teachers participating in the study theorized 
that the greatest factor in improved writing 
was related to the topic itself. The teachers 
felt that the 1987 topic—To What Extent 
Should the Natural Resources of the Eastern 
Slopes be Developed?—had little sense of 
immediacy or relevancy for many students. 
The commitment to a topic that lends itself to 
more involved argumentation and more 
persuasive writing was missing. 


In contrast, familiarity with the 1991 topic— 
How Should the Government of Alberta be 
Involved in Recycling Programs?—allowed 
students to tie their experiences and convic- 
tions to the writing assignment. Such a 
personal connection to the issue may partially 
account for the greater percentage of students 
able to achieve an Acceptable (3) score. That 
this broader 1991 topic was also a more acces- 
sible topic is evidenced by the greater percent- 
age of students responding to it. In 1987, 
3.8% of students produced INS papers, 
whereas in 1991 only 1% did: almost all 1991 
students felt competent to address this topic. 


A remarkably high percentage of students 
achieved an Acceptable (3) or higher score for 
Use of Language on the 1991 test. Teachers 
concluded that this might be attributed to two 
factors: 


*a greater emphasis on student writing in all 
subjects and at all grades levels 

*social studies classroom instruction and 
practice in effectively communicating ideas 
central to the development of issues 


Given that assignments have become 
cognitively more challenging, yet much more 


open-ended, and that standards have become 
more demanding and still students’ work has 
improved, one might conclude that classroom 
instruction that fosters critical thinking and 
clear, concrete expression of ideas has been 
the greatest factor in improving student 
writing. 


Detailed discussion of the study design, 
methodology, results, and conclusions can be 
obtained by calling the Achievement Testing 
Unit of Student Evaluation Branch, Alberta 
Education, at 427-0010. 
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SECTION 9 


DIRECTIONS: BROADENED ASSESSMENT 


PURPOSE 


Broadened assessment is a strategy that 
includes the use of different evaluation meth- 
ods and instruments designed to collect a 
broader range of information about what 
students know and can do. Along with paper 
and pencil tests, broadened assessment will 
assist us in creating a more complete picture 
of what students learn and how they apply 
those learnings. This strategy is designed to 
assess students in a manner that reflects the 
way students learn and to support effective 
classroom teaching. 


During the 1990-91 school year, the Achieve- 
ment Testing Unit of the Student Evaluation 
Branch initiated the development of many 
broadened assessment strategies. With the 
assistance of classroom teachers, the assess- 
ment instruments were validated for field 
testing purposes in the spring of 1991. The 
number of students who participated in each 
field test ranged from 150 to 1 350. A brief 
description of each instrument and some 
general findings based on student results 
follows. 


GRADE 3: ACROSS-CURRICULUM PORTFOLIO ASSESSMENT 


The portfolio project was designed to answer the question, At what level are Grade 3 
students performing in oral communication, problem-solving, participation, and 


process skills? 


ORAL COMMUNICATION SKILLS 


This component involved the audio taping 
and/or video taping of an oral presentation by 
Grade 3 students in classrooms across Al- 
berta. Students could make their presenta- 
tion individually or in small groups to the 
students in their classrooms. Presentations 
could range from puppetry or storytelling to 
interviewing or reporting but did not have to 
be language arts material specifically. 


Through an analysis of the audio and video 
tapes, we learned that students working at 
level show a move toward independence; 
however, occasional prompting and/or support 
is needed. These students have a beginning 
awareness of audience, purpose, the need to 
project their voices, and the importance of 
voice, face, and body language. The tasks 
they undertake are basic. Their props, when 
used, are simple, and they may have some 
difficulty using them. They are starting to 
show more confidence in their ability; hence, 
they are only somewhat nervous in front of an 
audience. 
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Students working beyond level demonstrate 
self-control and can work independently. 
These students need minimal support and are 
willing to take risks. They display confidence 
in their ability and have developed awareness 
of audience, purpose, and logical sequence. 
They recognize the importance of voice, face, 
and body language. 


Students not yet at level exhibited few of the 
characteristics of the students at level. They 
were reluctant participants, usually very 
nervous and intimidated by an audience. 


The teachers involved in this evaluation 
thought that the assessment of oral communi- 
cation skills in language learning has, for the 
most part, been neglected. They also thought 
that the basic premise of the study was a good 
one and suggested several modifications that, 
when incorporated, should result in an 
assessment instrument of improved quality. 


PROBLEM-SOLVING SKILLS 


This component was designed to gather infor- 
mation about Grade 3 students’ problem- 
solving skills and strategies. Teachers were 
asked to select, with their students, a prob- 
lem-solving activity and use Polya’s problem- 
solving model. The activity required the 
students to use insight, lent itself to a variety 
of solutions, and involved a mathematical 
process or skill. 


Teachers developed a holistic scoring guide to 
evaluate students’ work. 


We learned that most students performing at 
level could: 

¢ identify the question being asked 

* identify relevant and key words 


* develop a plan to find an answer 
¢ use the plan to find an answer 


Students performing beyond level could do 
all of the above as well as: 
* restate the problem in their own 
words 
* write a number sentence to match the 
problem 
® recognize that there is more than one 
way to solve a problem 
° identify and solve multiple-step 
problems 


Students not yet at level required constant 
prompting and support to engage in and 
complete the tasks. 


PARTICIPATION SKILLS 


Classroom teachers assessed their students’ 


participation skills through the use of the skill 


development charts found in the Social Stud- 
ies Program of Studies. Teachers agreed that 
the checklist approach would be the most 
effective and manageable way to collect the 
information. 


After the checklists were reviewed and the 
information was compiled, teachers agreed 
that students participating at level: 
¢ understand the need for rules and/or 
for observing them 
* often show respect for the rights and 
opinions of others 
¢ may be easily distracted but attention 
can be focused back 
¢ understand the difference between a 
leader and a follower 


Students participating beyond level had all 
of the above characteristics and: 
° are actively involved in making group 
rules 
° always show respect for the rights 
and opinions of others 


¢ stay focused despite distractions from 
others 
* accept a leadership role as necessary 


Those students not yet at level: 

° generally believe that rules are for 
others 

° are unable to stay on task 

¢ find it difficult to take turns and 
participate cooperatively 

¢ have difficulty understanding the 
difference between leader and fol- 
lower 


Teachers strongly support this method of 
assessing student participation skills because 
it supports the teaching and assessment of an 
important part of the curriculum that has 
been neglected. Teachers also suggested 
minor revisions to the checklist and the level 
descriptors in an effort to improve the quality 
and reliability in the information that this 
instrument gathers. 
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Process SKILLS 


This study required students to use materials 
provided in a baggie to complete seven tasks 
that were designed to assess their ability to 
perform process skills outlined in Division I 
Science. Analytic scoring criteria were devel- 
oped, validated, and applied by teachers. 
Results indicate that students performing at 
level could: 
* measure using arbitrary units and 
measure accurately 
¢ select appropriate devices for collect- 
ing data 
e predict from graphed data 
¢ classify natural objects according to 
only one property 


Student performing beyond level could also: 
° organize data in a graph 
¢ rank objects from heaviest to lightest 
¢ classify natural objects using more 
than one property 
* provide written descriptions of meth- 
ods of classification 


Students that were not yet at level could: 
* rank most objects from heaviest to 
lightest 
¢ classify some natural objects using 
only one property 


Teachers felt that this method of assessment 
was extremely useful and informative in that 
it: 
* supported classroom teaching 
* provided a meaningful context for learn- 
ing and assessment 
© provided results that could be used to 
improve instruction 


In addition, they said that students didn’t feel 
as though they were being tested. Instead, 
students commented that they were learning 
something as they progressed through each 
activity. 


GRADE 3: ACROSS-CURRICULUM THEMATIC TEST 


This field test focused on a theme integrating 
Grade 3 science, mathematics, social studies 
and language arts. New questions reflecting a 
context that students can easily relate to were 
included. The theme was chosen to reflect the 
way many students learn and teachers teach 
in the classroom. An eight-and-one-half-inch 
by fourteen-inch information sheet about the 
seasons provided context for the test and 
information for the student that could be used 
to answer each question. 


The questions were of different designs, 
ranging from shading in graphs to indicate 
the number of visitors to the park in winter to 
shading in the number of coins that would buy 
a toy boat worth 90 cents. For the writing 
assignment, students were able to make a 
choice. They could either write a report, using 
the facts provided, about the beaver and why 
beavers were good swimmers, or write a story 
about their favorite season, telling why it is 
their favorite. 
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The following are comments made by Grade 3 
teachers: 


“Reflects integration of subject areas— 
theme-based just like lessons.” 


“T think this is a great idea—it really fits in 
with the way primary education is done— 
thematically!” 


“This test is fun and interesting to do. I 
think children would enjoy this test and 
would find it challenging.” 


Only one teacher believed that this type of 
test was inappropriate. Students who wrote 
the test commented that it was interesting, 
hard, and made them think. Field testing this 
instrument resulted in valuable input from 
teachers and students that, when incorpo- 
rated into the test, will provide an assessment 
instrument of better quality. 


GRADE 6: MATHEMATICS AND SCIENCE PERFORMANCE-BASED ASSESSMENT 


This assessment was designed to answer the 
question, Can performance assessment 
broaden the description of what stu- 
dents know and can do in the applica- 
tion of their problem-solving and/or 
science process skills? Performance-based 
assessment engages students in the use of 
“hands-on" materials to complete meaningful 
real-life activities. Although there was no 
time limit, students were encouraged to 
complete each activity within 10 to 15 min- 
utes, 


Scoring criteria were used to describe what 
students demonstrated they knew and could 
do. Most students were able to: 


¢ infer possible effects of change within 

an ecosystem. 

infer the path that electricity travels 

in hidden circuits 

construct simple electrical circuits 

using a bulb, wire, and a dry cell 

classify animals as predator or prey 

on the basis of their special adapta- 

tions 

¢ make predictions and inferences 
based on direct observation and 
recorded data 

* locate the outline of a two-dimen- 
sional picture using ordered pairs 

¢ use manipulatives to determine the 
shortest route between two points on 
amap 


rank objects from heaviest to lightest 
using an equal-arm balance 

convert tally information to equiva- 
lent fractions and decimals 

construct bar graphs to accurately 
reflect data collected 

° create classifications for data collected 
¢ find and compare volume of rectangu- 
lar prisms 


Classroom teachers felt that this type of 
assessment provides accurate information 
about what students can actually do and how 
well they can do it. 


The following are comments made by stu- 
dents: 


“It was easy, big, and interesting.” 


“I thought that this was great, especially the 
science part where you had to join wires and 
make light bulbs work.” 


“In math I liked the K that you had to flip, 
slide, and turn.” 


“It’s a lot better than multiple-choice tests 
because you have things to work with in- 
stead of a paper and pencil.” 


After the teacher validation comments are 
reviewed further, some activities may have to 
be revised and perhaps new ones developed 
for field testing in spring 1992. 


GRADE 6; VIEWING AND LISTENING SKILLS ASSESSMENT 


This assessment was designed to answer the question, At what level are students in 
Grade 6 performing in listening and viewing skills? 


LISTENING SKILLS 


The listening skills component required 


students to listen to three pieces of literature. 


After hearing the fiction and nonfiction selec- 
tions, students answered some multiple- 


choice questions. After hearing the poetry 
piece, they retold what they heard. In addi- 
tion, students listened to some directions and 
did what the directions asked them to do. 


-74 - 


- 


The information collected was analysed by 
Grade 6 teachers. They also used information 
provided by teachers who field tested the 
instrument to describe student expectations 
for "at level" and "not yet at level" perform- 
ances. 


Students who are at level are able to retell a 
great many of the significant details and 
events. Most of the time, the sequencing of 
the events and/or details is correct. These 
students understand the denotative meanings 
of words they have heard; however, they may 
experience more difficulty with the connota- 
tive meaning of words. At level students can 
paraphrase a spoken message but will experi- 
ence some difficulty doing so. They can also 
follow oral directions but will make a few 
errors in the process. The at level students 
generally lack some of the confidence in their 
general listening skills typically found in the 
beyond level students. As a result, their 
critical listening skills are not quite so well 
developed. 


Students who are beyond level are consist- 
ently able to retell all significant details and 
events in proper order. These students 
understand both denotative and connotative 
meanings of words that they have heard. 
Beyond level students can paraphrase a 
spoken message with no difficulty and follow 
oral directions with no errors. In addition, 
they display confidence and competence in 
their general listening skills. As a result, 
they are excellent critical listeners as well. 


Students who are not yet at level are usually 
able to retell only a few of the significant 
details and events. Students functioning at 
this level have a confused idea of sequence. 
These students typically have a difficult time 
understanding all but the most basic vocabu- 
lary. Connotative meanings are not under- 
stood at all. Paraphrasing for these students 
is difficult because they probably had consid- 
erable difficulty understanding what they 
heard. 


VIEWING SKILLS 


For the viewing skills component of the as- 
sessment, students were given two pictures. 
They had to describe what they saw in each 
picture and then compare and contrast how 
they felt when they looked at both pictures. 
In addition, they had to look at a political 
cartoon and explain what message they got 
from it. 


The information collected was analysed by 
Grade 6 teachers. From the analysis, teachers 
were able to describe student performances 
that reflect curriculum expectations for at 
level, beyond level, and not yet at level 
performances. 


- 15 - 


Those students who are functioning at level 
are able to communicate in writing informa- 
tion they derive from a visual presentation; 
however, their statements tend to be general. 
The general knowledge base of these students 
is not as broad as those performing beyond 
level. When asked to describe a picture, these 
students are able to pick out general details, 
but the subtleties will escape their detection. 
When students performing at level examine 
pictures that display different emotions, they 
have some difficulty interpreting the various 
moods and feelings portrayed. The at level 
students can compare and contrast different 
pictures only if the pictures are relatively 
straightforward and not too detailed. 


Students who are functioning beyond level 
are able to communicate in writing informa- 
tion they derive from a visual presentation. 
The information is generally quite sophisti- 
cated because these student bring a broad 
general knowledge base to their understand- 
ing. When asked to describe a picture, stu- 
dents performing beyond level are able to 
pick out the most minute details. They are 
also able to examine pictures displaying 
different emotions and interpret the various 
moods and feelings portrayed. These beyond 
level students show an ability to compare and 
contrast different pictures, and provide many 
appropriate details. 


Students not yet at level demonstrate a 
limited understanding of what they view. 


Thus, they are able to communicate in writing 
in only a superficial manner. When asked to 
describe a picture, these students pick out 
only the most obvious features of the picture. 
Students not yet at level have a great deal of 
difficulty interpreting moods and feelings 
portrayed in pictures displaying various 
emotions. These students are not able to 
compare and contrast different pictures unless 
the pictures are very basic. 


This type of assessment was received very 
positively. In the spring of 1992, student 
viewing and listening skills will be assessed 
on a sample basis and reported next year with 
the Grade 6 English Language Arts Achieve- 
ment Test results. 


GRADE 9: MATHEMATICS AND SCIENCE PERFORMANCE-BASED ASSESSMENT 


This assessment was designed to answer the 
question, Can performance assessment 
broaden the description of what students 
know and can do in the application of 
their problem-solving and/or science 
process skills? Although there was no time 
limit, students were encouraged to complete 
each activity within 10 to 15 minutes. 


Scoring criteria were used to describe what 
students knew and could do. Students were 
able to: 


*find the square root of a number using 
estimation 

*understand and use a strategy for finding 
the surface area of any right prism 

*interpret maps and scale drawings 

*use ratios to solve problems 

*use a dichotomous key to classify animals 

¢determine experimentally which of two 
materials is a better insulation, which wall 
design conserves heat energy best, and 
which of two liquids is best to use in a cool- 
ing system 


*construct series and parallel electrical cir- 
cuits 

¢determine the effect of temperature on rates 
of reaction 


The performance of many Grade 9 students 
was disappointing. They experienced diffi- 
culty in arriving at solutions that were cor- 
rect. This may be the result of a weakness in 
lab skills and/or use of manipulative. 


However, Grade 9 students enjoyed the as- 
sessment and commented that it was better to 
have their hands actually on the materials to 
solve problems or explain concepts rather 
than imagine everything from a diagram on 
paper. Teachers saw ways in which these 
activities could be easily incorporated into 
classroom instruction. In addition, they saw 
how important aspects of the curriculum could 
be emphasized when incorporated into this 
type of assessment. 


- 16 - 


Se oe 


GRADE 9; PARTICIPATION SKILLS ASSESSMENT 


This assessment was designed to answer the 
question, At what level are students in 
Grade 9 performing in participation 
skills? Ten teachers from across Alberta 
used three different forms or checklists to 
indicate the intrapersonal skills, interper- 
sonal relations, and group process and discus- 
sion skills for each student. Accompanying 
the forms was a five-point rating scale, where 
1 indicated no evidence that the student 
possessed the skill and 5 represented student 
independence and control. 


The ratings assigned to students by teachers 
participating in the assessment provided a lot 
of detail about the participation skills of 
students in Grade 9. From this information, 
descriptions of at level, beyond level, and not 
yet at level performance of students’ partici- 
pation skills have been developed. 


Students performing at level deal well with 
minor frustrations and seek assistance when 
needed. These students often make good 
choices and have the ability to empathize with 
others. However, students performing at 
level tend to feel that they have the best 
opinion and sometimes are impulsive in 
responding. These students compromise 
occasionally and accept disagreement. In 
groups, they co-operate actively with others 
and work as directed by the teacher and/or 
group leader. Occasionally, students perform- 
ing at level will monopolize discussions. 


Students performing beyond level deal with 
personal frustration in a positive manner, 
seek help on more difficult assignments, and 
make logical choices. They also have an 
internal drive to challenge themselves. These 
students see how behavior affects others. 
They can be given a task and left to work at it 
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independently. In addition, beyond level 
students respond positively to the opinions of 
others and defend the rights of others. They 
wait to be recognized before speaking, spend 
time helping others freely, and seek compro- 
mise. Students performing beyond level 
display leadership skills in group discussions. 
They are always on task and lead others by 
example. 


Students performing not yet at level express 
anger and lack self-control when frustrated. 
They often refuse assistance and refuse to 
make decisions. These students criticize 
external forces and fail to see how their 
behavior can affect others. In many in- 
stances, these students monopolize their 
teachers’ time and require constant instruc- 
tion. 


Students not yet at level believe there is only 
one answer, and they will not compromise. 
They interrupt the class with negative or 
incorrect responses and attempt to redirect 
the flow of the class. These students respond 
negatively toward the mistakes of others and 
refuse to compromise or co-operate. They 
often use slang terms to express disagree- 
ment. 


In addition, students performing not yet at 
level seek to distract group members from the 
task at hand. If they don’t frequently and 
aggressively monopolize discussions, they 
totally withdraw from discussions. 


Participation skills of students are an impor- 
tant part of any program. With the assistance 
of classroom teachers, descriptions such as 
those presented here will continue to be 
revised and refined. 


FUTURE DIRECTIONS 


Broadened assessment will continue to be the 
focus for development and study in the future. 
The commitment of the Achievement Testing 
Unit is to search for assessment methods that 
will provide the opportunity for students to 
demonstrate what they have learned in other 
areas of the Programs of Study not normally 
assessed on a paper and pencil test. 


In 1992, broadened assessment instruments 
will be implemented as part of the regular 
achievement testing program and results will 
be reported as part of the overall results. 
Field tests will be administered in other areas 
as well: 


Grade 3 
*Participation Skills Assessment 
*Science Performance-based Assessment 
Language Learning Performance 
Assessment 


Grade 6 
° Listening and Viewing Skills Assessment 
°*Mathematics Performance-based 
Assessment 
*Science Performance-based Assessment 


Grade 9 
*Mathematics Performance-based 
Assessment 
Science Performance-based Assessment 
° Participation Skills Assessment 


For further information, please contact a 
member of the Achievement Testing Team: 


Greg Hall—Program Manager 

Valerie Doenz—Test Development Specialist, 
Social Studies 

Nola Aitken—Test Development Specialist, 
Mathematics 

Gérard Lavigne—Test Development Special- 
ist, French Language Arts 

Jo-Anne Hug—Test Development Specialist, 
English Language Arts 

Hank Boer—Test Development Specialist, 
Science 


Achievement Testing Program 
Student Evaluation Branch 
Alberta Education 

Devonian Building, West Tower 
11160 Jasper Avenue 
Edmonton, Alberta 

T5K OL2 


Phone: 427-0010 
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APPENDIX A 


STANDARDS 


The move toward a results-based curriculum has re-emphasized the need for a clear delinea- 
tion of standards and of their purpose. All standards and all methods of setting standards 
require judgment. 


The process of setting a standard can be only as good as the judgements that go into 
it. The standard will depend on whose judgements are involved in the process. In 
this sense, all standards are subjective. Yet, once a standard has been set, the 
decisions based on it can be made objectively. Instead of a separate set of judgements 
for each test-taker, you will have the same set of judgements applied to all test- 
takers. Standards cannot be objectively determined, but they can be objectively 


applied.' 


DEFINITIONS 


The Achievement Testing Program is directly 
concerned with three different but related 
standards. These standards are provincial 
curriculum standards, provincial achieve- 
ment standards, and provincial assessment 
standards. 


Curriculum Standards are the expected 
student learnings sequenced into levels. 
They include specific statements of knowl- 
edge, skill, and attitude expectations 
against which student performance is 
judged. These standards are established in 
the process of curriculum development and 
are found in the Program of Studies docu- 
ment produced for each course. 


Achievement Standards are judgments 
that specify what percentage of students 
are expected to achieve an acceptable and 
an excellent level of performance in rela- 
tion to each course of studies, i.e., the 
relevant curriculum standards. It is impor- 
tant to point out that this judgment is not a 
prediction of the percentage of students 
who will actually achieve acceptable or 
excellent levels of performance but rather 
is a specification of the percentage of stu- 
dents at a given age or grade in school who 
are expected to perform at an acceptable or 
excellent level. These standards apply to 
school, jurisdiction, and provincial achieve- 
ment. 


¢ Assessment Standards are the criteria 
adopted for judging actual student achieve- 
ment relative to curriculum standards. 
They are ultimately expressed and applied 
as test scores. They are derived from an- 
swers to the following or similar questions: 
What scores must students obtain or 
how many questions and/or which 
specific questions on a given test must 
students answer correctly in order for 
their performance on the test to be 
judged as acceptable (or excellent)? 
These standards apply to individual student 
performance. 


Indirectly, the Achievement Testing Program 
influences local targets. 


¢ Local Targets are the objectives set in 
schools and jurisdictions to assist students 
in moving toward or exceeding the provin- 
cial achievement standards. These local 
targets reflect the specific needs of individu- 
als and groups within a specific community. 


The Student Evaluation Branch is responsible 
for establishing and reporting the provincial 
assessment standards and shares responsibil- 
ity with the Curriculum Branch for establish- 
ing provincial achievement standards as they 
relate to the Achievement Testing Program. 


1Passing Scores; Samuel A. Livingston, Michael J. Zieky; Educational Testing Service, 1982 


-79 - 


PURPOSE 


The purpose of defining assessment and 
achievement standards, and the subsequent 
process of setting the provincial assessment 
standard within the sphere of the Achieve- 
ment Testing Program, is to answer questions 
such as: 


¢ What is “Acceptable” and “Excellent” per- 
formance in relation to the curricular expec- 
tations for students at the age or grade 
being tested? 

¢ What percentages of students at the age or 
grade being tested ought to achieve an 
“Acceptable” and “Excellent” level, assum- 
ing adequate teaching and resources? 

¢ What scores on a specific test shall reason- 
ably represent “Acceptable” and “Excellent” 
performance respectively? 


¢ What are the prevailing strengths and 
weaknesses of Alberta students in relation 
to the curriculum being tested? 

¢ What questions or tasks, when answered or 
performed correctly, on a particular test 
reflect “Acceptable” and “Excellent” per- 
formance? 


In essence, for each test and level of perform- 
ance, the assessment standards setters are 
challenged to answer the question, What 
score must a student obtain or how many 
questions must a student answer cor- 
rectly to be judged as having achieved an 
acceptable or excellent standard? Satis- 
factory performance may only be said to exist 
when the percentage of students scoring at or 
above the assessment standards is equal to or 
greater than the achievement standards. 


COMPONENT STANDARDS VERSUS A TOTAL TEST STANDARD 


Each achievement test has traditionally been 
divided into two major components. In 
mathematics, for example, the components 

are problem solving and subject matter. In the 
past, some students were achieving the ac- 
ceptable standard on the total test while being 
unable to perform well on one of the two major 
components. For example, a mathematics 
student might be able to answer correctly all 
or most of the subject matter questions but be 
very weak on the problem-solving questions. 
Based on a "total test score,” this student 
could have been reported as having achieved 
the acceptable standard in mathematics, 
whereas his or her performance on one major 


Figure A-1 


*Figure not to scale 


Grade 6 Mathematics Achievement Test 
Percentage of Students Achieving Standards by Major Component* 
N = 31 930 (100%) 


component would not have been acceptable. 
Using the Grade 6 Mathematics results to 
illustrate, Figure A-1 shows the percentage of 
students achieving the acceptable assessment 
standard for each of the two major compo- 
nents. Note that the percentage of students 
achieving the acceptable standard for both 
components is less than the percentage 
achieving the acceptable standard on each of 
the components. Some students achieved the 
acceptable standard on only one of the major 
components. Students, now and in the future, 
who are judged to have acceptable perform- 
ance will have met the standard on both 
components. 


16.9% 
Did not achieve 
acceptable standard 
on either component 
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Tables dealing with standards in sections 2, 3, 
and 4 of this report show what percentage of 
students achieved standards on each of the 
major components and what percentage of — 
students achieved standards on both compo- 
nents. For this year only, a total test stand- 
ard was also determined and reported in 
order to provide for comparison with previous 
results. 


It should be noted that standards were set 
orily on the English versions of the achieve- 
ment tests and they apply only to students in 
regular English programs. The Francophone, 
French Immersion, and other programs did 
not have sufficient numbers to set standards 
in a fair and equitable manner. Students in 
these programs should not be compared with 
the standards set for the majority English 
program population; however, local target 
standards are recommended for these groups. 


THE ASSESSMENT STANDARD-SETTING PROCESS 


Figure A-2 shows the model initiated by the 
Student Evaluation Branch to set standards 
for the 1991 achievement tests. The objec- 
tive was to widen the process of setting 
assessment standards as much as possi- 
ble over previous years and especially to 


Figure A-2 


provide for community input and feed- 

back. The process involves individuals and 
groups in making judgments that contribute 
to establishing the assessment and achieve- 

ment standards. 


Process Model for Standard Setting 
Achievement Testing Program 
June 1991 


INPUT 


2 
Technical Review 
Committee 


Public Advisory 
Committee. 


4 
Teacher Standards 
Committee 


6 


1 Director's 5 


Curriculum and 
Test Developers 
Committee 


Final Standards Review 
Committee 


Analytic Services 
Committee 
Student Evaluation 


Representatives 
from 1, 2, 3,.4,5 


PRODUCT 


ASSESSMENT STANDARDS 
Raw scores (cut-off scores) representing 
Acceptable Standard and 
Standard of Excellence 


an 
ACHIEVEMENT STANDARDS 
Percentage of students expected to achieve 
Acceptable Standard or Higher and 
Standard of Excellence 
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Except for the members of the Public Advisory 
Committee, the members of all committees 
outlined in Figure A-2 are expected to be 
highly knowledgeable about both the curricu- 
lum and the learning characteristics of the 
students who are writing the tests. The com- 
mittees are briefly described below. 


1. Curriculum and Test Developers 
Committee 


There is one Curriculum and Test Developers 
Committee for each achievement test. The 
committee consists of Alberta Education 
consultants, curriculum developers, and test 
development specialists. Their objective is to 
recommend assessment standards to the Final 
Standards Review Committee. They may also 
review and make recommendations on 
achievement standards. 


2. Technical Review Committee 


There is one Technical Review Committee for 
each achievement test. Experienced and 
knowledgeable educators—teachers, superin- 
tendents, university professors, and Alberta 
Education consultants—meet with the devel- 
opers of each test. Their purpose is to review 
the test and make recommendations for 
improvements where necessary. Because of 
time constraints, the Technical Review Com- 
mittee was not formally involved in the set- 
ting of assessment standards for June 1991. 
However, in future years this committee will 
set and recommend assessment standards. In 
addition, it will review the appropriateness of 
current course achievement standards. 


3. Public Advisory Committee 


The Public Advisory Committee, composed of 
representatives from the general public, met 
and discussed standards for the first time this 
year. See Appendix B for more information on 
the proceedings of this committee. 


2A Consumer’s Guide to Setting Performance Standards on Criterion-Referenced Tests, Ronald A. Berk: Review of 


Educational Research, Spring 1986, Volume 56. 
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4. Teacher Standards Committee 


Approximately 20 experienced teachers from 
different areas of the province are selected to 
sit on a Teacher Standards Committee for each 
test. To be selected for a committee, a teacher | 
must have been teaching in the grade and 
subject area for the previous two years. At 
present, the procedures used to aid teachers in 

setting the assessment standards are the 

modified Angoff method for Mathematics and 

Science subjects and the Nedelsky method for 
Humanities subjects.? The teachers make 

judgments about appropriate assessment 

standards and make recommendations to the 

Final Standards Review Committee. As well, 

they review achievement standards for appro- 
priateness. 


5. Analytic Services Committee 


Evaluation Branch is the professional quality 
control and advisory group for the complete 
process of standard setting. There is a com- 
mittee of professionals in psychometrics and 
statistics for each test. A critical function is to 
ensure that Alberta Education’s standard- 
setting procedures produce technically valid 
results that meet the strict requirements of 
high quality professional studies. The commit- 
tee makes recommendations for assessment 
standard-setting procedures, leads individual 
group discussions, and recommends improve- 
ments where necessary. Another function is to 
determine, independently, assessment stand- 
ards through statistical analyses of current 
student achievement data.’ 


| 
i 
The Analytic Services Unit of the Student | 
| 
| 


6. Final Standards Review Committee 


The Final Standards Review Committee con- 
sists of representatives from the above com- 
mittees and is chaired by the Director of 
Student Evaluation. In separate sessions, the 
recommendations of all test committees are 
presented and, through consensus, final as- 
sessment standards are adopted for each test. 


1991 ASSESSMENT AND ACHIEVEMENT STANDARDS 


To achieve an acceptable standard, students answer correctly a minimum number of 


were required to obtain a certain score or to questions from the larger group of 

answer correctly a certain number of ques- specific questions that were identified by 
tions from any part of the test. To achieve a standard setters as measuring achievement at 
standard of excellence in previous years, an excellent level. 


students were only required to answer cor- 

rectly a certain number of questions from the Tables A-1, A-2, and A-3 show the assessment 
total test. In 1991, for the first time, students and achievement standards adopted in June 
were required to satisfy an additional crite- 1991 by the Final Standards Review Commit- 


rion. To achieve a standard of excellence in tee for the grades 3, 6, and 9 achievement 
grades 3 and 6, students were required to tests. 
Table A-1 


Grade 3 Science 
Assessment and Achievement Standards 
June 1991 


Provincial Provincial 
Maximum Assessment Achievement 


Possible Standard Standard 
Raw Score (Raw Score) (Per Cent) 


Standard of Excellence 
Process Skills 


Subject Matter 

Both Major Components 
Acceptable Standard 

Process Skills 

Subject Matter 

Both Major Components 


*The 32 correct answers must include at least nine of the 13 questions that measure achievement at an excellent 
level. The 13 question numbers are 5, 6, 13, 16, 22, 35, 37, 38, 39, 44, 45, 48, and 50. These are 13 of the 36 
Process Skills questions identified as questions that would challenge students performing at a level of excellence. 


**The 12 correct answers must include both of the questions that measure achievement at an excellent level. The two 
question numbers are 2 and 17. These are two of the 14 Subject Matter questions identified as questions that 
would challenge students performing at a level of excellence. 
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Table A-2 
Grade 6 Mathematics 
Assessment and Achievement Standards 
June 1991 


Provincial 
Achievement 
Standard 

(Per Cent) 


Provincial 
Assessment 
Standard 

(Raw Score) 


Standard of Excellence 
Problem Solving 
Subject Matter 
Both Major Components 

Acceptable Standard 

Problem Solving 

Subject Matter 

Both Major Components 


*The 21 correct answers must include at least nine of the 11 problem-solving questions that measure achievement at 
the excellent level. The 11 question numbers are 4, 19, 25, 27, 32, 41, 50, 51, 52, 53, and 54. 


Table A-3 
Grade 9 Social Studies 
Assessment and Achievement Standards 
June 1991 


Provincial 
Achievement 
Standard 

(Per Cent) 


Provincial 
Assessment 
Standard 

(Raw Score) 


Standard of Excellence 
Facts, Concepts, and 
Generalizations 
Process Skills 
Both Major Components 
Acceptable Standard 
Facts, Concepts, and 
Generalizations 
Process Skills 
Both Major Components 


During 1991, much review and discussion of standards was undertaken before adjusting the 
process of standard setting to meet the current and impending needs of Alberta Education’s 
Achievement Testing Program. The exercise was enlightening and useful. However, we do not 
expect that the process will fit the needs of all. We invite your questions, comments, and 
recommendations. Please send your observations to Michael Robinson, Assistant Director, 
Analytic Services, Student Evaluation Branch, at 427-2948. 


| 
re 
| 
| 
| 


APPENDIX B 


PUBLIC REVIEW OF ACHIEVEMENT STANDARDS 
AND RESULTS 


On July 24 and 25, 1991, representatives of Mathematics 6 

nine business, professional, and community 

organizations reviewed achievement test Students should be able to: 

results and discussed standards. This was * know how to solve (at minimum) two-step 
the first time such a meeting was held. It problems 

was planned to ensure that input from groups * perform straightforward numerical opera- 
representing "public" interests would be tions (addition, subtraction, multiplica- 
considered in the overall reporting of results. tion, division) 

The purpose of the meeting was to gather * interpret and use information using 
direct public input into the question, How various methods (e.g., graphs) 

well should Alberta students achieve in ¢ estimate in different situations 

Grade 3 Science, Grade 6 Mathematics, * collect and record data 


and Grade 9 Social Studies? 

Social Studies 9 
The two-day meeting was designed to collect 
information that would contribute toa better | Students should be able to: 


understanding of how well our students in ¢ think critically to support decision- 
Alberta are doing. This was accomplished by making skills (interpret data, make 
exploring and discussing answers to four connections, integrate ideas, etc.) 
questions. Each question is listed below, * present both sides of an issue clearly, with 
along with the results of the meeting. supporting information 
* understand the structure of Canadian 

¢ What do members of the public expect society and related social issues 

students to know and be able to do in * use good communication skills (oral and 

Grade 3 Science, Grade 6 Mathematics, written) 

and Grade 9 Social Studies? * use good research skills 
Comments: For each subject area, partici- ¢ How closely do the achievement tests 
pants listed the knowledge and skills that reflect the expectations described by 
they felt were important learnings for stu- the group? 


dents to achieve at each grade level. 
Participants reviewed the achievement tests 
The following key points highlight the group’s and selected questions they felt students 


views: should be able to answer correctly to demon- 
strate an acceptable level of performance. 
Science 3 Through discussion, the knowledge and skills 
that students require to answer these ques- 
Students should be able to: tions successfully were listed. The knowledge 
¢ demonstrate a curiosity toward science and skills listed were all very close to the 
¢ make decisions and take risks when original expectations held by the group. 


solving problems 
* use science skills in real-life situations 
* communicate both orally and in writing 
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One area the group thought was important 
but was not addressed in any of the achieve- 
ment tests was students’ attitudes toward 
learning. Concern was also expressed that 
student communication skills were not evalu- 
ated in the mathematics and science tests. At 
minimum, a written component would be 
needed to do this. The group was not unani- 
mous about whether written skills in science 
and mathematics should be evaluated if they 
were already evaluated in language arts and 
social studies. 


The group stated that the writing assignment 
in social studies was an excellent way to 
evaluate student knowledge and skills. The 
assignment presented an issue and required 
students to state and support a position of 
their choice. 


e How well did students actually perform 
on the achievement tests? 


Results showing how well students performed 
on important multiple-choice items were 
presented to the group. As well, a first draft 
of the written-response analysis for social 
studies was presented. 


In all subject areas, students met some of the 
group’s standards; in some areas of learning, 
achievement was below the group’s standards. 
Some of these areas are listed below. 


Science 3 


Met Standards 
Performance on the test indicated that stu- 
dents: 
¢ know basic plant and animal life in Al- 
berta 
° are able to make deductions that require 
reasoning 


Below Standards 
Performance on the test indicated that many 
students have difficulty: 
° making judgments related to cause and 
effect 


¢ understanding interrelationships in the 
environment 

¢ making practical applications and solving 
problems 


Mathematics 6 


Met Standards 
Performance on the test indicated that stu- 
dents: 
© can perform operations (multiplying, 
dividing, adding, subtracting) 
° can perform one-step problems 


Below Standards 
Performance on the test indicated that many 
students have difficulty: 
° using information and skills to solve two- 
step problems 
° computing simple averages 


Social Studies 9 


Met Standards 
Performance on the test indicated that stu- 
dents: 
* understand points of view 
¢ know centrally planned, mixed, and 
market economies from a basic (or 
definitional) perspective 


Below Standards 
Performance on the test indicated that many 
students have difficulty: 
¢ understanding the role of labor, the 
entrepeneur, and the government in the 
different economies 
¢ distinguishing the interrelationship of 
social, economic, and political factors in 
the different economies 
° writing effectively when choosing and 
supporting a position 


Jian 


¢ Did the group feel that students’ per- 
formance on the achievement tests was 
good enough? 


Generally, the group felt that, given their 
expectations for both levels, student perform- 
ance was not good enough. 


Several other comments were made during 
the two-day meeting: 


¢ All participants indicated they had had an 
adequate opportunity to express their views. 


¢ Participants viewed the activity/discussion 
approach as an excellent meeting format. 


¢ On many occasions, different participants 
stated that expectations and standards have 
to be high to ensure our children are reach- 
ing for high goals. 
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° More time to delve into each subject would 
have yielded more specific commentary on 
expectations/learnings. 


¢ This type of meeting should be continued to 
permit ongoing input from groups represent- 
ing public interests. 


Finally, because of the value of this meeting, 
and the desire to continue gathering public 
input into the standards and expectations 
that exist for student achievement, a similar 
meeting will be held in the summer of 1992. 


A more detailed report on the 1991 meeting 
was distributed to participants and is avail- 
able upon request. Please contact Marcia 
Wright at 427-0010 to obtain a copy of this 
report. 


APPENDIX C 


GUIDELINES TO INTERPRETATION OF RESULTS 


Following each administration of the achieve- 
ment tests, a Provincial Report is prepared. 
This report is a public document that de- 
scribes the aggregated results obtained by 
those students who wrote achievement tests 
in a given year. Provincial reports can be 
used by school board members, superintend- 
ents, principals, and teachers as they review 
results for students in their own jurisdictions 
and schools. By using the Provincial Report 
in this way, policymakers and educators can 
check their perceptions of local achievement 
against provincewide standards and trends in 
the levels of achievement. 


This Provincial Report describes the re- 
sults achieved by students who wrote the 
June 1991 achievement tests in Grade 3 
Science, Grade 6 Mathematics, and Grade 9 
Social Studies. 


This year, for the first time, jurisdiction 
and school reports were sent to superin- 
tendents and principals before the Provincial 
Report was ready for distribution. Policy- 
makers and educators in each jurisdiction are 
encouraged to study carefully the provincial 
results and their own test results. 


It should be noted that results for groups of 
fewer than 25 students should be interpreted 
with caution. Results for such groups can be 
unduly affected by the presence of a few 
extreme scores when the analyses are per- 
formed to produce the respective group statis- 
tics. Consequently, any generalizations may 
appear to indicate performance that is higher 
or lower, better or worse, than it is in relation 
to the criteria applied. 


Educators at the school and jurisdiction levels 
can make three kinds of comparisons to decide 
if the achievement of their students is "good 
enough.” One comparison is in relation to 
expectations or standards; another is in 
relation to the achievement of students in the 
entire province of Alberta; and the third is to 
look at the jurisdiction results in 1987, when 
tests in these subjects were last administered. 
After making these comparisons, teachers, 
principals, and superintendents can identify 
the strengths in the programs that were 
delivered in their grades 3, 6, and 9 class- 
rooms. Where the results show weaknesses, 
changes can be made where possible and 
desirable. 


USE OF THE REPORTS 


The reports are not intended to be used as the 

basis for 

*making decisions about student placement 
or promotion 

*evaluating teacher performance 

*comparing performance between or among 
schools. 


Administrators in each jurisdiction should 
apply separate locally developed teacher, 
school, and school system evaluation policies 
to the tasks of evaluating teacher and school 
performance. 


The information provided in the reports is 
factual regarding what the test results are. 
The interpretation of this information— 
hypothesizing why results are as they are— 
involves consideration of the many factors and 
variables that contribute to achievement. 


In addition, it must be noted that the informa- 
tion in these reports is limited to selected 
objectives of the Program of Studies. Many 
important aspects of learning cannot be 
measured by the time-limited, paper and 
pencil achievement tests. 
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ASSESSMENT STANDARDS AND ACHIEVEMENT STANDARDS 


Standards were confirmed for each achieve- 
ment test and are reported both here in the 
Provincial Report (sections 1, 2, 3, and 4 and 
Appendix A) and in tables 2 and 3 of the 
jurisdiction and school reports. Criteria 
representing a standard of excellence and 
an acceptable standard were confirmed for 
each of the two major reporting categories 
(components) of the test. These criteria are 
referred to as assessment standards. The 
assessment standards are supplemented by 
achievement standards, which specify expec- 
tations for the percentage of students in the 
province expected to meet the assessment 
standards. Appendix A provides more detail 
on both the standards and the process by 
which they have been confirmed. 


Standards are primarily reported in figures 
1-2 to 1-5, 2-1, 2-2, 3-1, 3-2, 4-1, 4-2, and 
tables 2-3, 2-4, 3-4, 3-5, 4-4 and 4-5 of the 
Provincial Report and tables 2 and 3 of each 
school and jurisdiction report. These tables 
show the number of students writing the test 
who met or exceeded the provincial achieve- 
ment standards, the number who were ex- 
pected to meet or exceed the standards, and 
the percentages who met the standards on 
each or both of the two major components of 
the test. 


Table 3 of each school and jurisdiction report 
gives the same information for the total test 
rather than by major component. 


The school and jurisdiction tables also show 
whether the number of students in that school 
or jurisdiction who achieved the standard is 
significantly different from the expected 
number. A 95% confidence interval is used for 
these purposes; this criterion means that 
differences are only reported when there is a 
5% or smaller probability that a difference of 
that size could occur by chance. For schools 
and jurisdictions with fewer than five or six 
students, significances cannot be calculated, 
and the indications are omitted. Although the 
statistical tests take the number of students 
into consideration, it is a useful rule of thumb 
that results for groups of fewer than 25 stu- 
dents must be interpreted with particular 
caution. Chance variation in small groups is 
greater. 


Educators interpreting these reports are 
encouraged to consider how well their stu- 
dents have done compared to the assessment 
standards and how well schools and jurisdic- 
tions have done compared to the achievement 
standards. 


CoMmPARING RESULTS TO AVERAGE SCORES 


Overall test results are presented in relation 
to provincial standards. Tables 4 and 5 of 
each jurisdiction and school report also pro- 
vide jurisdiction or school average scores for 
each reporting category or subtest. Each of 
these scores may be compared to the provin- 
cial average for the same reporting category 
or subtest to determine if differences exist. 
Tables 2-7, 3-8, and 4-8 of the Provincial 
Report provide similar information. 


The importance of differences that may exist 
between jurisdiction or school averages and 
provincial averages is not always clear. To 
aid in the interpretation of differences be- 
tween the averages, jurisdiction and school 
reports indicate when differences are signifi- 
cant using a 95% confidence interval. 
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FACTORS LIMITING THE INTERPRETATION OF TEST RESULTS 


Educators who are interpreting results must 2. If more than 10% of eligible students in a 


take into account the following limitations: jurisdiction did not write a test, the re- 
ported statistics for that jurisdiction may 
1. Paper and pencil tests necessarily measure not accurately represent the true level of 
reading achievement in the content area achievement. 


being tested. Standards built into each 
achievement test reflect the reading level 3. Consideration should be given to the 


expectation for the grade level tested. degree to which students in particular 
Jurisdictions should consider the average classes or grades were motivated to per- 
reading level of their grades 3, 6, and 9 form to their level of ability. 


students, as reading levels below these 
grades will have an effect on test results. 


FACTORS THAT MAY AFFECT STUDENT ACHIEVEMENT 


Results on the achievement tests may have 3. Availability and Appropriateness of 


been affected by: Resources 
* programs of study 

1. Environment *curriculum guides 
*community environment *resource materials 
*school environment including staff morale library services 
*socioeconomic background *current textbooks 
*family circumstances *references 

2. Personal Factors 4. Instruction 
* ability *teacher qualifications (i.e., to teach a 
* attitude particular subject) 
* motivation *teacher experience 
*aspiration *teacher morale 
*academic background *professional development 
*learning style ¢staff turnover 


¢ professional support 
*teaching strategies 
*timetabling constraints or influences 
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A SYSTEMATIC APPROACH TO THE EFFECTIVE USE OF TEST RESULTS 


Achievement test results can be used con- 
structively as one means of improving the 
quality of education. A systematic use of the 
test results would include the following steps: 


*individual written-response question 
results (when appropriate). 
2. Noting any patterns, anomalies, and/or 
interrelationships in the results. 
1. Comparing test results for a jurisdiction or 


school to the provincial results. Be sure . Hypothesizing relationships between your 


that your comparisons include the 

°total test score, 

°total and subtest scores for multiple- 
choice questions, 

*total and subtest scores for written 
response assignments (when appropri- 
ate), 

*individual multiple-choice question 
results, and 


4. 


observations and any of the factors listed 

in these guidelines that may have had an 

effect on achievement or achievement test 
results. 


Considering and implementing a plan that 
will help to improve the quality of educa- 
tion for students. 


AN ADMINISTRATIVE MODEL FOR THE EFFECTIVE USE OF 
ACHIEVEMENT TEST RESULTS 


The following model may be useful for those who wish to develop a constructive system for 
interpreting achievement test results. 


Basic PRINCIPLES 


. Itis desirable and feasible for teachers 
and school administrators to make use of 
achievement test results in analysing the 
performance of their own students. 


It is more constructive for schools to 
develop their own analyses, interpreta- 
tions, and action plans than to have these 
imposed externally. 


. The impact of factors such as those listed 
in these guidelines should be analysed and 
discussed when reviewing achievement 
test results. 


. Subtest or reporting category results are 
usually more informative than total test 
scores. 


5. 
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Generalizations should be stated with 
caution and should be supported by evi- 
dence that is independent of achievement 
test results. 


It is neither desirable nor productive to 
compare the results of different schools. 


Achievement tests measure many of the 
objectives specified by the curriculum. 
However, skills and concepts that are not 
measured by the achievement tests are 
also to be taught and evaluated at the 
local level. 


Staff discussions as well as written reports 
are useful means of ensuring that results 
are appropriately interpreted and used. 


SUGGESTED CONTENT FOR INTERPRETATION OF INDIVIDUAL SCHOOL RESULTS 


Subject, grade level, and date of achieve- 
ment test administration 


. Number of students who wrote the 


achievement test 


. Profiles of students or groups who wrote 
the achievement test, which include 
—previous performances 

—number of students repeating the grade 


. Program emphases, such as hours of 
instruction, skill and content emphases 


Instructional practices, such as methodol- 
ogy, resources, and the relationship be- 
tween the program offered and the provin- 
cial curriculum 


6. Program objectives not measured by the 


10. 


11. 


achievement test 


. School results compared with provincial 


results on subtests . 


. Current school results compared with 


those of previous administrations 


. Discussion of item results, identification of 


common student errors, and suggestions of 
ways for reducing the misunderstanding 
that leads to these errors 


Recommendations for the following year 
or semester 


Summary and concluding comments 


SUGGESTED REPORTING STRUCTURE 


. Teachers and/or the principal analyse the 
results and prepare a written report about 
each administration of an achievement 
test. 


. The principal reviews and signs the report. 


. The report is shared with central office 
supervisory personnel. 


. The appropriate central office supervisory 
personnel prepare a written response to 
the report, with copies of the response 
going to the teachers and the principal. 


5. 
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If possible, all involved staff meet to 
discuss the report and the response. 


Reports are used to improve the program 
and maximize future opportunities for 
student success. 


When large differences exist between 
expected and actual achievement test 
results over time, consideration should be 
given to conducting a formal program 
evaluation. 


APPENDIX D 


REPORTING TO PARENTS 


What are the achievement tests? 


The achievement tests are provincial govern- 
ment tests administered in Alberta schools to 
grades 3, 6, and 9 students in language arts, 
social studies, science, and mathematics. 


What is the purpose of the achievement 
tests? 


The achievement tests help Alberta Education 
to communicate provincial expectations and 
results for levels of student performance in 
language arts, social studies, science, and 
mathematics. The tests enable Alberta Edu- 
cation to monitor the level of achievement of 
students throughout Alberta. The results also 
help local school board, principals, and teach- 
ers identify the strengths and weaknesses in 
their implementation of these subjects. 


How many achievement tests will my 
child have to write? 


Students write only one achievement test in 
Grade 3, one in Grade 6, and one in Grade 9. 
Tests are rotated so that a different subject is 
tested each year. In 1991, Grade 3 students 
wrote the science achievement test, Grade 6 
students wrote the mathematics achievement 
test, and Grade 9 students wrote the social 
studies achievement test. In 1992, Grade 3 
students will write the social studies achieve- 
ment test, Grade 6 students will write the 
language arts achievement test, and Grade 9 
students will write the mathematics achieve- 
ment test. 


How should I prepare my child to write 
an achievement test? 


No preparation beyond normal classroom 
instruction is required to write an achieve- 
ment test. While students should be encour- 
aged to do their best, a good night’s sleep and 


a relaxed, confident approach to testing are 
the best possible preparation. 


How much do these tests count for my 
child? 


The achievement tests do NOT affect stu- 
dents’ final marks. The classroom teacher is 
responsible for evaluating students and 
awarding final marks. Achievement test 
results are not released by Alberta Education 
until October, long after students’ marks have 
been determined by the classroom teacher. 


How do achievement test results help 
classroom teachers? 


Achievement test results provide feedback on 
student achievement to school boards, princi- 
pals, and teachers. For example, teachers in a 
school that consistently scores high on one 
part of the curriculum but low on another may 
wish to examine their programs to see if 
changes are needed to achieve a better in- 
structional balance. 


What are the limitations of the achieve- 
ment tests? 


Paper and pencil tests cannot easily measure 
such things as laboratory skills, small group 
discussions, and creative thinking. Thus, 
some student strengths can be evaluated only 
by the classroom teacher. Also, a single test 
cannot reveal as much about a student’s 
development and growth as can evaluation by 
the classroom teacher over the course of a full 
school year. 


What advantage do achievement tests 
have over other standardized tests? 


Unlike commercially developed tests, achieve- 
ment tests are based specifically on Alberta’s 
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programs of study and are designed, written, 
and evaluated by experienced classroom 
teachers from across the province. Tests 
developed elsewhere may not reflect curricu- 
lum or standards appropriate for Alberta. 


How do I interpret achievement test re- 
sults? 


The Achievement Testing Program Provincial 
Report includes guidelines for interpreting 
results. Readers are cautioned not to overgen- 
eralize conclusions based on a single admini- 
stration of the test. Results should be com- 
pared with expectations or with the results of 
previous achievement tests in the same sub- 
ject. Any trends that are observed in the 
scores must then be interpreted in the context 
of a variety of factors that could affect student 
achievement, such as the school and commu- 
nity environment, students’ socioeconomic 
background, and available learning resources. 


Comparisons between districts, schools, or 
classrooms are likely to prove misleading and 
are therefore discouraged. 


Can I find out how my child did on the 
achievement test? 


Individual results for the achievement tests 
are made available to school principals. Since 
the tests are designed to gather information 
on groups of students and not on individuals, 
individual results must be interpreted with 
caution. 


Where can I get additional information 
about the Achievement Testing Program? 


Bulletins describing the content of the coming 
year’s achievement tests and the Provincial 
Report describing the results of the previous 
year’s testing are distributed to schools each 
fall. Requests for copies of these publications 
or questions and comments regarding the 
Achievement Testing Program should be 
directed to: 


Mr. Dennis Belyk 

Assistant Director 

Achievement Testing and Diagnostic 
Evaluation Programs 

Student Evaluation Branch 

Alberta Education 

Devonian Building, West Tower 

11160 Jasper Avenue 

Edmonton, Alberta T5K OL2 
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APPENDIX E 


DEVELOPING ACHIEVEMENT TESTS 


The Student Evaluation Branch develops 
achievement tests that measure student 
achievement at the grades 3, 6, and 9 levels. 
Provincewide testing in language arts, math- 
ematics, science, and social studies follows a 
four-year cycle for each grade level and sub- 
ject. Many individuals and groups are in- 
volved in the development of each test: prac- 
tising classroom teachers, school and central 
office administrators, and representatives of 
postsecondary institutions, the Curriculum 
Design Branch, the Language Services 
Branch, Regional Offices, and the Student 
Evaluation Branch. Student Evaluation 
Branch staff ensure that valid and reliable 
tests are developed. 


The following is a summary of the phases of 
the test development process: 


¢ Planning 
¢ Approving Blueprints 
¢ Developing Test Questions 


¢ Constructing and Administering 
Field Tests 


e Analysing and Revising 

¢ Constructing Final Field Tests 
¢ Approving Final Field Tests 

¢ Administering Final Field Tests 
¢ Constructing the Final Test 


¢ Preparing and Administering the 
Final Test 


* Marking 


¢ Analysing and Reporting the Results 
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Under normal circumstances, it takes three 
years to complete all phases of the process. 


PLANNING 


Test developers ensure that the design of each 
achievement test reflects the goals and objec- 
tives of the Program of Studies and the cur- 
riculum specifications for each subject. Plan- 
ning takes into consideration those parts of 
the program that are testable in a paper and 
pencil format within a given time frame. 
Teachers and consultants from across the 
province assist in preparing the design of each 
test. 


Test developers prepare an interim test 
blueprint (an overall plan used to guide the 
development of a test). Questions that must 
be addressed at this point are: 


¢What knowledge and skills should students 
be expected to possess? 


¢What types of questions will constitute the 
test (multiple choice, short answer, or ex- 
tended written response)? 


¢What weighting will each part of the test be 
given? 


¢ How long and how demanding should the 
test be? 


¢How should the results of the test be organ- 
ized for reporting purposes? 


In order to ensure that each test will produce 
meaningful and reliable results, test develop- 
ers incorporate statistical as well as curricular 
standards in the test design. Statistical 
standards include projected test means, range 
of question difficulty, and the requirements 
for reporting. For example, the ideal mean of 
a multiple-choice test containing questions 


with four alternatives is 62.5%. This is the 
midpoint between chance selection (25%) and 
perfection (100%). The range of difficulty of 
multiple-choice questions is expected to vary 
from 30% to 85% to ensure that students with 
varying ability levels are challenged. 


Each dimension of the curriculum for which 
results are reported must contain at least six 
questions if the results are to be meaningful. 


APPROVING BLUEPRINTS 


Blueprint approval establishes the overall 
design of the test, the exact emphases given to 
each category for which results are reported, 
and the emphases given to the different 
cognitive levels. 


The interim blueprint is reviewed by a com- 
mittee of Alberta Education personnel that 
represents the Curriculum Design Branch (or 
Language Services Branch), Regional Office 
consultants, and the Student Evaluation 
Branch. This committee makes recommenda- 
tions to the Director of the Student Evalua- 
tion Branch. 


The blueprint recommended by the Alberta 
Education committee is then reviewed by a 
Test Review Committee, which consists of 
members nominated by the Alberta Teachers’ 
Association, the Conference of Alberta School 
Superintendents, postsecondary institutions, 
and Alberta Education. This committee 
makes recommendations to the Director of the 
Student Evaluation Branch. 


DEVELOPING TEST QUESTIONS 


Following blueprint approval, committees of 
practising classroom teachers working at the 
appropriate grade level are formed, and 
question development meetings are held. 
These committees develop new test questions 
that reflect the goals and objectives of the 
Program of Studies and curriculum specifica- 
tions. Where necessary, question developers 
are trained in the principles of question 
construction. Questions built in committee 


are then screened for format, validity, blue- 
print ‘fit’, and other design considerations. 


CONSTRUCTING AND ADMINISTERING 
FIELD TESTS 


After careful editing and formatting of ques- 
tions developed by the teacher committees, 
field tests are constructed. Any required 
artwork is completed during this phase of the 
test development process. 


With permission from school and jurisdiction 
personnel, field tests are sent to a number of 
teachers throughout Alberta. The students 
involved are representative of the student 
population for which the test has been de- 
signed. A minimum sample of 150 students 
writes each field test. 


Teachers who administer a field test are 
asked to comment in writing on the following: 


*reading level 


¢how closely the question matches the way in 
which a concept was taught 


*level of difficulty of the questions 
¢quality of the questions and graphics 
*errors of any kind 


The results from the administration of this 
initial round of field tests are used to validate 
content, to determine difficulty levels, and to 
ensure that questions are expressed clearly. 
Special field tests are also constructed to “try 
out” new assessment strategies and tech- 
niques that may be useful for future assess- 


ments. 


ANALYSING AND REVISING 


The results of each field test are then ana- 
lysed and scrutinized to determine whether 
individual questions require revision. 
Teacher comments regarding the way that 
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test questions are structured and the way that 
a subject is being taught are also carefully 
considered and used to guide revision. 


Questions deemed to require changes are 
revised and submitted for further field test- 


ing. 


CONSTRUCTING FINAL FIELD TESTS 


Once the initial field test results are thor- 
oughly analysed and questions requiring 
changes are revised, final field tests are 
constructed. These field tests follow the 
approved blueprint and parallel the actual 
achievement test in format and design. 


Final field tests, like all field tests, are sub- 
mitted for further validity checking, editing, 
and proofreading. In grades 6 and 9, separate 
tests in English and in French are developed 
for language arts. At this point, all other tests 
for Grade 6 and Grade 9 are translated into 
French. 


APPROVING FINAL FIELD TESTS 


After the final field tests have been con- 
structed, a second meeting of the Alberta 
Education Committee that represents the 
Curriculum Design Branch (or Language 
Services Branch), Regional Office consultants, 
and the Student Evaluation Branch is con- 
vened. This committee reviews the final field 
tests and makes recommendations for im- 
provement. 


The Test Review Committee, which approved 
the blueprint in phase two of the test develop- 
ment process, meets a second time to review 
and recommend for approval the final field 
tests and the instructions for administering 
the tests. If a test includes short-answer or 
extended-writing questions, the Test Review 
Committee discusses standards of achieve- 
ment and marking standards appropriate for 
the test. Again, this committee makes recom- 
mendations to the Director of the Student 
Evaluation Branch. 
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ADMINISTERING FINAL FIELD TESTS 


The final field tests are administered and the 
results are used as a final screen in selecting 
questions for placement on the provincial 
achievement test. A minimum sample of 250 
students writes each final field test. The 
sample is selected to include: 


© only students who have received instruction 
in the course 


students representing a normal distribution 
of ability levels 


students from rural and urban schools 


students from large and small schools 


CONSTRUCTING THE FINAL TEST 


The construction of the final test form is based 
upon information collected from the final field 
test administration. The Test Review Com- 
mittee is reconvened to review the final test 
form. 


The test is submitted for final validity check- 
ing, editing, and proofreading. Grade 6 and 
Grade 9 achievement tests, in subjects other 
than language arts, are translated into 
French. 


For each test, an information bulletin is 
prepared that outlines the design and nature 
of the upcoming tests. These bulletins are 
distributed to each school in September to 
facilitate program and instructional planning 
by teachers and administrators. 


PREPARING AND ADMINISTERING THE 
FINAL TEST 


The completed achievement test is commer- 
cially printed and prepared for distribution. 
Sufficient copies of the test are mailed to each 
school. Quantities are based on the number of 
students enrolled in the subject as reported to 
the Student Evaluation Branch. The test is 
administered to students by their classroom 
teachers. 


MARKING 


All written-response sections of the tests are 
marked by classroom teachers. These teach- 
ers, who are recommended by their superin- 
tendents, are currently teaching the course 
being evaluated, have taught the course for a 
minimum of two years, and hold a valid 
Alberta Permanent Professional Certificate. 
Student Evaluation Branch staff train and 
supervise the teachers during the marking 
sessions. All multiple-choice responses are 
machine scored. 


ANALYSING AND REPORTING THE RESULTS 


A results report is prepared and distributed to 
superintendents, school principals, Alberta 
Education officials, and other Departments of 
Education. This report is also made available 
to the general public. In addition to the 
Achievement Testing Program Provincial 
Report, each school and jurisdiction receives a 
statistical summary for its student population. 


For further information, please refer to the 
Achievement Test Bulletins, or call the Assist- 
ant Director, Achievement Testing and Diag- 
nostic Evaluation Programs, at 427-0010. 
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ACHIEVEMENT TESTING PROGRAM 


PROVINCIAL REPORT QUESTIONNAIRE 


The Student Evaluation Branch strives to Please take a moment to respond to the fol- 
produce documents that will be useful to the lowing questions. Then detach this sheet and 
educational community. The purpose of the send it to: 

following questionnaire is to collect your 

opinions about the Provincial Report. These = Mr. Michael Robinson 

opinions will be considered when the content Assistant Director, Analytic Services 


and format of the report are reviewed before Student Evaluation Branch 
June 1992. Alberta Education 
11160 Jasper Avenue 


Edmonton, Alberta T5K 0L2 


USE OF THE REPORT 


Please check the boxes that apply to you. 


1. My present role is primarily that of 


* teacher = 
¢ school administrator [ ] 


¢ central office administrator = 


* school board member [] 


* other (please specify) 


2. [read the report, but I DID NOT use it to interpret the results attained by my students. Py 


3. [read the report, and I used it to interpret the results attained by the students in 


* my classroon * my school * my jurisdiction 


[J = [ 


Please respond to the following statement if you have checked one of the three boxes in question 3. 
4. Ihave made use of the results to alter the educational program offered in 


* my classroon * my school * my jurisdiction 


a LJ [J 


Continued 
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CONTENT OF THE REPORT 


Please judge the usefulness of the information included in the various sections of the report by checking the 
appropriate boxes below. 
Very Adequate Of Some Of No 
Useful for Use Use Use 
Section 1: Summary of Achievement Test Results 
Section 2: Grade 3 Science 
Section 3: Grade 6 Mathematics 
Section 4: Grade 9 Social Studies 
Section 5: Achievement By Gender 
Section 6: Achievement By Age 


Section 7: Achievement by Grade Level 


Section 8: Achievement Over Time 


HU UUUUUUE 
WU UUUUUUU 
UU UUUUUUU 
WU UUUUUUU 


Section 9: Directions: Broadened Assessment 


If you wish, please comment further on the content of the report in the space below. 


FORMAT OF THE REPORT 


Please judge the usefulness of the report’s format by checking the appropriate boxes below. 
Very Adequate Of Some 


Organization into Separate 
Sections 


Double-Column Presentation 
of Text 


Presentation of Figures 
Presentation of Tables 


Blending of Information 
in Text, Figures, and Tables 


NUUU Us 
UG OG OU #8 


If you wish, please comment further on the format of the report in the space below. 
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